Adding beauty products to augmented reality tutorials

ABSTRACT

Systems, methods, and computer-readable media for adding beauty products to tutorials are presented. Methods include accessing video data comprising images of a presenter creating a tutorial, the tutorial depicting the presenter applying a beauty product to a body part of the presenter. Methods further include processing the video data to identify changes to the body part of the presenter from an application of the beauty product, and responding to identifying changes to the body part of the presenter from the application of the beauty product by processing the video data to identify the beauty product. Methods further include retrieving information regarding the beauty product and causing presentation of information regarding the beauty product on a display device.

CLAIM OF PRIORITY

This application is a continuation of U.S. patent application Ser. No.17/155,715, filed on Jan. 22, 2021, which claims the benefit of priorityto U.S. Provisional Application Ser. No. 62/705,076, filed on Jun. 10,2020, which are incorporated herein by reference in their entirety.

BACKGROUND

The application of beauty products has become more and more complicated.Often there are multiple steps in applying a beauty product.Additionally, often the user of the beauty product does not know how toapply the beauty product. Users of beauty products often rely on goingto salons or beauty counters for help in applying beauty products, butthis may be expensive, or it may be difficult to get a person that isexperienced in a particular beauty product. Moreover, applying a beautyproduct may be both expensive and time-consuming, and the user may bedisappointed in the results.

Video-based tutorials and training programs have recently emerged toprovide users with training in any number of skills, including theapplication of beauty products. There are a number of technicalchallenges that exist with respect to making these tutorials andtraining programs engaging, and also presenting students or viewers withsufficient information so that the skills can be easily absorbed andmastered. Additionally, there are a number of technical challenges inenabling presenters to create tutorials without the creation of thetutorial becoming tedious and time-consuming.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

To easily identify the discussion of any particular element or act, themost significant digit or digits in a reference number refer to thefigure number in which that element is first introduced.

FIG. 1 illustrates an augmented reality (AR) tutorial system, inaccordance with some embodiments.

FIG. 2 illustrates components of AR tutorial data, in accordance withsome embodiments.

FIG. 3 illustrates components of beauty product data, in accordance withsome embodiments.

FIG. 4 illustrates the operation of an AR tutorial system, in accordancewith some embodiments.

FIGS. 5A, 5B, and 5C illustrate the operation of an AR tutorial, inaccordance with some embodiments.

FIG. 6 illustrates a flow diagram for using an AR tutorial, inaccordance with some embodiments.

FIG. 7 illustrates a point of entry, in accordance with someembodiments.

FIG. 8 illustrates an AR tutorial carousel, in accordance with someembodiments.

FIG. 9 illustrates points of entry into AR tutorial, in accordance withsome embodiments.

FIG. 10 illustrates a point of entry, in accordance with someembodiments.

FIG. 11 illustrates a point of entry, in accordance with someembodiments.

FIG. 12 illustrates capturing an image, in accordance with someembodiments.

FIG. 13 illustrates capturing an image, in accordance with someembodiments.

FIG. 14 illustrates captured images, in accordance with someembodiments.

FIG. 15 illustrates information for a beauty product, in accordance withsome embodiments.

FIG. 16 illustrates steps of an AR tutorial, in accordance with someembodiments.

FIG. 17 illustrates moving to a next step, in accordance with someembodiments.

FIG. 18 is a block diagram showing an example of an AR tutorial system,in accordance with some embodiments.

FIG. 19 is a block diagram illustrating further details regarding the ARtutorial system, according to example embodiments.

FIG. 20 is a schematic diagram illustrating data structures that may bestored in the database of the AR tutorial server system, according tocertain example embodiments.

FIG. 21 is a diagrammatic representation of the machine within whichinstructions (e.g., software, a program, an application, an applet, anapp, or other executable code) for causing the machine to perform anyone or more of the methodologies discussed herein may be executed.

FIG. 22 is a block diagram illustrating a software architecture, whichcan be installed on any one or more of the devices described herein.

FIG. 23 illustrates components of editing an AR tutorial, in accordancewith some embodiments.

FIG. 24 illustrates components of adding a step in an AR tutorial, inaccordance with some embodiments.

FIG. 25 illustrates components of creating a pause in an AR tutorial, inaccordance with some embodiments.

FIG. 26 illustrates components of adding a body part, in accordance withsome embodiments.

FIG. 27 illustrates components of adding a body part, in accordance withsome embodiments.

FIG. 28 illustrates components of adding a body part, in accordance withsome embodiments.

FIG. 29 illustrates components of adding a beauty product, in accordancewith some embodiments.

FIG. 30 illustrates components of adding a beauty product, in accordancewith some embodiments.

FIG. 31 illustrates components of adding or editing motion, inaccordance with some embodiments.

FIG. 32 illustrates a system for creating an AR tutorial, in accordancewith some embodiments.

FIG. 33 illustrates a method of adding beauty products to AR beautyproduct tutorials, in accordance with some embodiments.

FIG. 34 illustrates components of determine beauty product module, inaccordance with some embodiments.

FIG. 35 illustrates components of adding beauty products to augmentedreality tutorials, in accordance with some embodiments.

FIG. 36 illustrates components of adding beauty products to ARtutorials, in accordance with some embodiments.

FIG. 37 illustrates information regarding a beauty product, inaccordance with some embodiments.

FIG. 38 illustrates information regarding a beauty product, inaccordance with some embodiments.

FIG. 39 illustrates components of adding beauty products to augmentedreality tutorials, in accordance with some embodiments.

FIG. 40 illustrates components of adding beauty products to ARtutorials, in accordance with some embodiments.

FIG. 41 illustrates components of adding beauty products to ARtutorials, in accordance with some embodiments.

FIG. 42 illustrates components of scanning a beauty product, inaccordance with some embodiments.

FIG. 43 illustrates previewing an AR tutorial, in accordance with someembodiments.

FIG. 44 illustrates displaying beauty products associated with an ARtutorial, in accordance with some embodiments.

FIG. 45 illustrates components of scanning a beauty product, inaccordance with some embodiments.

DETAILED DESCRIPTION

An application enables a user (e.g., presenter) to edit and prepare abeauty product augmented reality (AR) tutorial for a user of the ARtutorial. The presenter may create an AR tutorial with multiple stepswhere different beauty products are used in different steps. Theapplication detects when a presenter applies a beauty product to a bodypart of the presenter and identifies the beauty product.

The application assists the presenter in identifying the beauty productby identifying the effects of applying the beauty product to a body partof the presenter and using this information to assist in identifying thebeauty product. The effects of applying the beauty product andinformation associated with the beauty product may be used to determinewhat effects should be used on a user of the AR tutorial to simulate theapplication of the beauty product to the user. The application mayassist the presenter in identifying the beauty product by detecting anew beauty product in video data of the presenter. The application mayassist the presenter in identifying the beauty product by identifyingwhen the presenter is applying a new beauty product and to which bodypart the presenter is applying the beauty product.

The application responds to commands from the user to identify thebeauty product where the commands may be hand gestures, voice command,or a selection of a user interface item. The application may indicatethat the presenter needs to reposition the beauty product to assist theapplication in identifying the beauty product. The application mayidentify information associated with the beauty product that may includeinformation that is custom to the presenter such as a website thatoffers the beauty product for sale to users of the AR tutorial.

The AR tutorial provides a dual-screen view with a recorded version ofthe presenter on one screen and a live view of the user on the otherscreen. The AR tutorial determines a body part of the presenter wherethe beauty product is being used and matches that to a body part of theuser. For example, if the presenter is applying eye shadow, then thebody part is the eye. The AR tutorial will zoom to the eye of the userof the tutorial. The AR tutorial determines the effects of the beautyproduct on the presenter. For example, if the presenter is applying eyeshadow, then the AR tutorial determines the change in the color to thepresenter's skin around the eye. The AR tutorial also determines themotion the presenter used to apply the beauty product. For example, themotion of a brush around the presenter's eye to apply the eye shadow.

The AR tutorial applies the effects to the live view of the user on theother screen. For example, AR tutorial will change the color of the skinaround the eye of the user. Additionally, the AR tutorial indicates themotion the presenter used in applying the beauty product on the bodypart of the user. For example, the AR tutorial may illustrate themovement of a brush on the presenter with white lines around the user'seye. The AR tutorial will synchronize the illustration of the effectsand movement on the live view of the user with the application of thebeauty product by the presenter.

Some embodiments provide a technical solution to the technical problemof how to identify a beauty product from live images of a presenterrecording a tutorial using the beauty product. The AR tutorial may useinformation associated with the beauty product to perform AR functionswithin the AR tutorial as disclosed herein. Some embodiments provide atechnical solution to the technical problem of how to identify beautyproducts used in a tutorial based on determining a body part the beautyproduct is used on, within the AR tutorial. Some embodiments provide atechnical solution to the technical problem of how to identify beautyproducts used in tutorials based on determining the effects the beautyproduct has on a body part of a presenter of AR tutorial. Someembodiments provide a technical solution to the technical problem ofstoring and retrieving customized and generic information for a beautyproduct by creating a unique identifier for the beauty products withinthe AR tutorial that include a generic identifier for the beautyproduct.

The user of the tutorial may be able to customize the beauty product.For example, the user may be able to use a different color of eyeshadow. The user of the AR tutorial is offered an opportunity topurchase the beauty products. In an example, the tutorial provides atool such as Lenses®, media overlays, and so forth that allows the userto preview the end results of the AR tutorial on the live view of theuser before the user starts the AR tutorial.

FIGS. 1-4 are disclosed in conjunction with one another. FIG. 1illustrates an augmented reality (AR) tutorial system 100, in accordancewith some embodiments. FIG. 2 illustrates components of AR tutorial data202, in accordance with some embodiments. FIG. 3 illustrates componentsof beauty product data 302, in accordance with some embodiments. FIG. 4illustrates the operation 400 of an AR tutorial system, in accordancewith some embodiments.

The AR tutorial module 102 is configured to generate an AR tutorial 143.In one example, the AR tutorial module 102 is part of a computingsystem, such as AR tutorial server system 1808 shown in FIG. 18 . Insome embodiments, the functionality of AR tutorial module 102 isimplemented with machine 2100 and/or an apparatus or portion of machine2100. The AR tutorial module 102 sends to and receives commands 112 froma mobile device 146, such as commands to-from 136. The AR tutorialmodule 102 receives live images 134 from the mobile device 146 andgenerates AR tutorial images 138 and AR user images 140 based on ARtutorial data 202 and beauty product data 302. The AR tutorial module102 is configured to generate AR tutorial images 138, AR user images140, an AR tutorial data 202, and/or a beauty product data 302. The ARtutorial module 102 is configured to be executed by one or moreprocessors of the mobile device 146, in accordance with someembodiments. The AR tutorial module 102 is configured to cause therendering of the AR tutorial display 142 and/or the AR user display 144,in accordance with some embodiments. Live images 134 are received by theAR tutorial module 102, from the camera 130, in accordance with someembodiments.

Mobile device 146 is a mobile computing device such as a smartphone,watch, tablet, glasses, or other portable computing devices. Mobiledevice 146 is a client device 1802A or client device 1802B as disclosedin conjunction with FIG. 18 , in accordance with some embodiments.Mobile device 146 is a machine 2100, in accordance with someembodiments. The mobile device 146 is a wireless device in communicationwith a wireless communication network, such as LTE, 5G, 3 GPP orInstitute of Electrical and Electronic Engineering (IEEE) 802.11, and soforth. In some embodiments, the mobile device 146 is a laptop computeror desktop computer. The camera 130 is an environmental component 2132,in accordance with some embodiments. The screen 132 is a display screenof the mobile device 146. The screen 132 is a user output component 2124and/or a user input component 2126, in accordance with some embodiments.The screen 132 is termed a display device, in accordance with someembodiments. The screen 132 is a portion of AR glasses or a projectionon an eye or eyes of a user of AR tutorial 143, in accordance with someembodiments. In some embodiments screen 132 is used to refer to the userinterface provided on the screen 132. Full screen indicates that AR userdisplay 144 or AR tutorial display 142 occupies both the screen 132portion of AR user display 144 or AR tutorial display 142 and all of orpart of the AR tutorial display 142 or AR user display, respectively, inaccordance with some embodiments. As disclosed in conjunction with FIG.32 , computing device 3208 may be a mobile device 146.

Commands to-from 136 are commands 112 to the AR tutorial module 102 fromthe mobile device 146 and commands 112 sent from the AR tutorial module102 to the mobile device 146. Live images 134 are images from the camera130, which may be a single image or a series of images of a video. ARtutorial display 142 is the portion of the screen 132 where the ARtutorial images 138 are presented. AR user display 144 is the portion ofthe screen 132 where AR user images 140 is displayed. As illustrated,the AR tutorial display 142 is shown in a top portion of the screen;however, it is to be understood that the AR tutorial display 142 couldbe shown in the bottom portion of the screen, or on a side portion ofthe screen, in example embodiments. Additionally, other information maybe displayed on the screen 132. In some embodiments, one of AR tutorialdisplay 142 and/or AR user display 144 may be displayed in a full screenof screen 132. In some embodiments, information may be considered ontop, under, to left, or to the right eye of the screen 132, and may beaccessed by a command 112 such as a swipe from the user 238 of FIG. 2 .

Determine motion module 122 extracts tutorial motion 216 from the ARtutorial video 228. Determine motion module 122 generates AR motion 217from the tutorial motion 216 of the AR tutorial video 228, in accordancewith some embodiments. In some embodiments, determine motion module 122generates tutorial motion 216 from live images 134. For example, as anAR tutorial video 228 is being captured the determine motion module 122extracts tutorial motion 216. Determine motion module 122 may beconfigured to determine tutorial motion 216 based on object recognitionwhere images of beauty product 224 of FIG. 2 , tutorial effects 218,and/or a body part 222 are identified in the live images 134 and/or ARtutorial video 228. Determine motion module 122 compares successiveimages to determine the difference between the images, in accordancewith some embodiments. For example, as illustrated in FIGS. 5A, 5B, and5C, determine motion module 122 detects the movement of the beautyproduct 224B, from screen 500A to screen 500B and then from screen 500Bto screen 500C. The result may be a measure of an amount of movement ofthe beauty product 224A for a number of images. In some embodiments,determine motion module 122 compares successive images of the ARtutorial video 228, which is being played on the screen 500A, screen500B, and screen 500C. Object recognition is used to determine thelocation of the beauty product 224B within the images of the AR tutorialvideo 228 and then a difference in position is calculated. For example,a number of pixels of movement of the beauty product 224B may bedetermined from screen 500A to screen 500B. As an example, data oftutorial motion 216 may include 2 pixels to the right for a differencebetween two successive images of the AR tutorial video 228.

In some embodiments, determine motion module 122 determines the motionof the presenter 236 to determine how the beauty product 224 is appliedto the presenter 236. Determine motion module 122 uses a neural networkto identify different portions of the images of AR tutorial video 228,in accordance with some embodiments. Determine motion module 122 uses aneural network to determine a difference between two images and may usetutorial effects 218 to determine the motion of the beauty product 224.For example, if a new portion of an eye has a different color in asecond image from a first image, then the tutorial motion 216 may bedetermined to include a position of the beauty product 224 in a firstimage position of AR tutorial video 228 to a position in a second imagethat would enable the presenter 236 to apply the beauty product 224 togenerate the different color between the first image and the secondimage.

Determine motion module 122 is configured to determine AR motion 217from tutorial motion 216. For example, determine motion module 122 isconfigured to determine AR motion 217 of FIGS. 2, 4, 5A, 5B, 5C, and 12from tutorial motion 216. FIGS. 5A, 5B, and 5C illustrate an example ofadding AR motion 217A, 217B, 217C, to AR user image 140 as displayed onAR user display 144 from tutorial motion 216A, which are the indicationsof motion of a beauty product 224A in AR tutorial video 228. FIGS. 5A,5B, and 5C illustrate the synchronized 434 movement of beauty product224A, 224B, and 224C by presenter 236 with the addition of AR motion217A, 217B, and 217C, respectively. In some embodiments, tutorial motion216 is predetermined. In some embodiments, tutorial motion 216 indicatesa number of pixels of motion and direction per image of the AR tutorialvideo 228.

Determine motion module 122 may determine a starting position in thelive images 134 of the user 238 for the AR motion 217. Additionally,determine motion module 122 adjusts the starting point of AR motion 217for body part 222, zoom level 208, and visible area 206. The user 238may be moving during the playing of the AR tutorial video 228, sodetermine motion module 122 is configured to adjust motion indicationsto compensate for the motion within the live images 134 of the user 238.As an example, as disclosed in conjunction with FIG. 5A, determinemotion module 122 has determined AR motion 217A based on the tutorialmotion 216A for the first “0:12” seconds in the AR tutorial 143 asdisplayed at time in tutorial 530A. Determine motion module 122 thenmust determine AR motion 217B for the difference between “0:12” secondsand “0:17” seconds as displayed at time in tutorial 530B of FIG. 5B.Determine motion module 122 may determine the AR motion 217B based on apredetermined number of images or a predetermined time to generate newAR motion 217B to be displayed by generate user images module 108. Insome embodiments, a fixed number of images are displayed in the AR userdisplay 144 such as 30 frames per second, and determine motion module122 provides a movement line for each frame. A movement line mayindicate pixels within the frame that should be indicated as movement.The movement line may be represented as a set of pixels, a vector, aline equation, or another representation. Determine motion module 122 isconfigured to enable the presenter 236 to edit the tutorial motion 216such as is disclosed in conjunction with FIG. 31 and herein.

Determine body part module 124 determines a body part 222 from ARtutorial video 228 and/or user video 230. In some embodiments, beautyproduct data 302 includes body part 308, which indicates a body part,that may be used by determine body part module 124 in determining a bodypart 222. The presenter 236 may enter the body part 222 via a userinterface (UI). In some embodiments, determine body part module 124 usesa neural network that is trained to identify different body parts froman image of a human body. Determine body part module 124 may use otherinformation to determine which body part 222 is having the beautyproduct 224 applied. For example, the determine body part module 124 maydetermine that an eye region has changed colors in an AR tutorial video228 or in images from live images 134 as the AR tutorial video 228 isbeing recorded. Determine body part module 124 is configured to identifya body part 222 from AR tutorial video 228 of the presenter 236 and usethe identified body part 222 of the presenter to determine the body part222 in the user video 230 of user 238. For example, as disclosed inconjunction with FIG. 4 , the determine body part module 124 canidentify body part 222A, which is a right eye, of the presenter 236 andthen identify a body part 222B, which is a right eye of a user 238, inthe user video 230. Determine body part module 124 is configured todetermine body part 222 as disclosed in conjunction with FIGS. 26-28 and29 , in accordance with some embodiments. Determine beauty productmodule 126 is disclosed in conjunction with FIG. 34 .

Determine effects module 128 determines the tutorial effects 218 basedon comparing images of the AR tutorial video 228. For example, determineeffects module 128 may compare successive images of the AR tutorialvideo 228 to determine a difference between the images and determinethat the difference is due to the application of the beauty product 224.Tutorial effects 218 may be data that indicates the difference betweensuccessive images due to the application of the beauty product 224, inaccordance with some embodiments. Determine effects module 128determines tutorial effects 218 using one or more of the following:tutorial motion 216, body part 222, and/or beauty product data 302 suchas color 316. Determine effects module 128 may use other data todetermine the tutorial effects 218. Determine effects module 128 maydetermine the differences between images on the body part 222 or bodypart 308. Body part 222 and body part 308 may indicate a same part bodypart. In some embodiments, body part 222 may be different than body part308, e.g., body part 308 may be from a manufacturer, such as is includedin beauty product information 304, and may indicate an eyelid, whereasbody part 222 may be more generic and indicate a right eye or left eye.Body part 222 and body part 308 may be different in other ways.Determine effects module 128 may determine the differences betweenimages on the portion of the image where motion 216 occurred between thetwo images of AR tutorial video 228. In some embodiments, tutorialeffects 218 are predetermined. As disclosed in conjunction with FIGS. 4,5A, 5B, 5C, 12, and 17 the determine effects module 128 determines thetutorial effects 218. Determine effects module 128 may determined thetutorial effects 218 further based on effects 334 which indicatesexpected changes or changes to the body part from applying the beautyproduct 224. In some embodiments, determine effects module 128 mayaccess video data such as AR tutorial video 228 and process the videodata to determine the tutorial effects 218.

In some embodiments, the determine effects module 128 is configured todetermine the AR effects 219 of FIGS. 2, 4, 5A, 5B, 5C, 8, 11, 12, 13,and 14 from tutorial effects 218 and/or AR tutorial video 228. FIGS. 5A,5B, and 5C illustrate an example of adding AR effects 219A, 219B, 219C,to a body part 222 that is a right eye of a user 238 from tutorialeffects 218A, which are the changes to a right eye of a presenter 236from applying beauty product 224B with beauty product 224A. FIGS. 5A,5B, and 5C illustrate the progression of the application of the eyeshadow where the changes to the right eye of the user 238 with the AReffects 219A, 219B, and 219C are synchronized with the changes to theright eye of the presenter 236, which are indicated by tutorial effects218A, 218B, and 218C. In some embodiments, the determine effects module128 uses deep learning that indicates changes to the live images 134that should be made based on the AR tutorial video 228. In someembodiments, AR effects 219 are determined based beauty productinformation 304 that indicates changes that the beauty product 224 ofbeauty product data 302 will make to the body part 222, 308. Forexample, a color such as color 316 may be indicated as the change thatis made to the user 238 from the application of the beauty product 224.AR effects 219 may be determined based on the color and an area of bodypart 222 or body part 308 to apply the color to the live images 134 ofthe user 238. In some embodiments, AR effects 219 are determined basedon skin tone where a skin tone of the user 238 is determined and thenthe application of the beauty product 224 is determined based on theskin tone of the user 238.

The generate tutorial images module 104 generates the AR tutorial images138. AR tutorial images 138 are used to generate the AR tutorial display142. Generate tutorial images module 104 generates AR tutorial images138 based on processing AR tutorial video 228. Generate tutorial imagesmodule 104 generates or plays the AR tutorial video 228 for a user 238to take the AR tutorial 143 with the AR tutorial data 202. Generatetutorial images module 104 may process the AR tutorial video 228 toinclude one or more of the following: beauty product 224, play bar, suchas play bar 422 of FIG. 4 , playback speed bar 414, playback speed value210, play bar 422, current play 424, step marks 426, back 428, capture432, UI items 234, and so forth. Generate tutorial images module 104generates AR tutorial images 138 based on playback speed value 210, inaccordance with some embodiments. Generate tutorial images module 104generates AR tutorial images 138 based on the zoom level 208 and visiblearea 206. For example, the AR tutorial video 228 may include a video ofa head of the presenter 236 and generate tutorial images module 104generates or processes the AR tutorial images 138 based on the zoomlevel 208 and visible area 206 of the AR tutorial video 228 comprisingvideo of the head of the presenter 236. For example, as illustrated inFIG. 4 , a right eye of presenter 236 is illustrated in AR tutorialdisplay 142, but AR tutorial video 228 may be of the whole head of thepresenter 236. Generate tutorial images module 104 is configured togenerate the AR tutorial images 138 as disclosed herein in conjunctionwith FIGS. 1-23 . In some embodiments, the generate tutorial imagesmodule 104 is configured to generate AR tutorial images 138 that arerelated to the AR tutorial 143 prior to the user 238 entering the ARtutorial 143. In some embodiments, generate tutorial images module 104is configured to generate AR tutorial images 138 that are related to theAR tutorial 143 prior to the user 238 entering the AR tutorial 143, suchas AR tutorial 606 under capture 832 of FIG. 8 , screen 800, and/orscreen 1100, or as part of image capturing such as screen 1200B, screen1300, screen 1400, and/or screen 1450.

The generate user images module 108 generates AR user images 140 thatare displayed in AR user display 144. The generate user images module108 renders or generate AR user display 144, in accordance with someembodiments. The generate user images module 108 processes live images134 to incorporate AR effects 219 and/or AR motion 217, in accordancewith some embodiments. For example, AR effects 219 may indicate an areato change a color of the live images 134 as disclosed in conjunctionwith FIGS. 5A, 5B, and 5C. In another example, AR motion 217 mayindicate pixels to add indications of motion on the live images 134 asdisclosed in conjunction with FIGS. 5A, 5B, and 5C. The generate userimages module 108 adds UI items 234 to the live images 134, inaccordance with some embodiments.

The generate user images module 108 processes the live images 134 basedon one of more the following: visible area 206, zoom level 208, playbackspeed value 210, tutorial motion 216, tutorial effects 218, body part222, and/or customization 226. In one example, the generate user imagesmodule 108 zooms to an identified body part 222. For example, a user 238may be capturing a video of a full face with live image 134 and, asillustrated in FIG. 4 , the generate user images module 108 zooms to thebody part 222, such as the eye of the full face of the user 238.

In some embodiments, AR motion 217 and/or AR effects 219 indicateschanges to make on a frame by frame basis of the live images 134.Generate user images module 108 may add items, such as UI items 234 tothe AR user images 140. Some example items that can be added includecapture 432, 832, play bar 422, playback speed bar 414, step marks 426,current play 424, back 428, full screen 1216, send 1212, added effects1306, added effects 1406, and the like. The generate user images module108 generates AR user display 144, in accordance with some embodiments.Generate user images module 108 is configured to generate AR user images140 as disclosed herein. Tutorial effects 218, tutorial motion 216,visible area 206, zoom level 208, playback speed value 210, pause 220,body part 222, beauty product 224, and/or customization 226 may havestart times 212 and end times 214 that indicate a time within the ARtutorial video 228 when the parameter should be applied by generate userimages module 108. The functionality performed by generate user imagesmodule 108 and/or generate tutorial images module 104 is termedprocessing the images, in accordance with some embodiments.

Respond to commands module 110 is configured to respond to commands 112from the mobile device 146 and/or user 238 via the mobile device 146.Commands 112 include pause, play, forward 1604 and back 1602 of FIG. 16, changing a playback speed value with a playback speed bar 414 of FIG.4 , selecting a capture 432, selecting a beauty product 224A, 224B toget additional information, adjusting a zoom level, done editing, editmotion, add beauty product, confirm, beauty product selection, add bodypart, add pause, add step, and so forth. Respond to commands module 110is configured to respond to commands 112 as disclosed herein. Commands112 may be indicated by the user 238 in different ways, such as byspeech, finger motion on the screen 132, motion of user 238 in the liveimages 134, such as opening a mouth, making an OK sign, use of a mouse,movement of an virtual reality headset, and so forth. In one example, atouch of finger 810 will pause the AR tutorial 143 and another touch ofthe finger 810 will play the AR tutorial 143. In another example, movinga thumb and finger 810 will adjust a zoom level 208.

The synchronize effects and motion module 106 synchronizes AR tutorialimages 138 with AR user images 140. For example, as illustrated in FIGS.5A, 5B, and 5C (and discussed herein) synchronize effects and motionmodule 106 synchronizes the displaying of tutorial effects 218A, 218B,218C with AR effects 219A, 219B, 219C, respectively, and tutorialmotions 216A, 216B, 216C with AR motions 217A, 217B, 217C, respectively.The synchronize effects and motion module 106 may be given live images134 with AR motion 217 and AR effects 219 added by the generate userimages module 108 and given AR tutorial images 138 from the generatetutorial images module 104 and synchronize the playing of the ARtutorial images 138 with the playing of the AR user images 140. In someembodiments, generate user images module 108 synchronizes AR user images140 with AR tutorial images 138.

Edit AR tutorial module 158 is configured to respond to commands 112 toedit the AR tutorial 143. Edit AR tutorial module 158 works inconjunction with responds to commands module 110, in accordance withsome embodiments. For example, edit AR tutorial module 158 is configuredto respond to edits of zoom level 208, playback speed value 210, andvisible area 206 as disclosed in conjunction with FIG. 23 . Edit ARtutorial module 158 is configured to respond to commands to delete astep, move a step, and/or add step 2406, such as step N 204, asdisclosed in conjunction with FIG. 24 . Edit AR tutorial module 158 isconfigured to respond to commands to add pause 2504 such as pause 220 asdisclosed in conjunction with FIG. 25 . Edit AR tutorial module 158 isconfigured to add body part 222 as disclosed in conjunction with FIGS.26-28, and 30 . Edit AR tutorial module 158 is configured to add abeauty product 224 as disclosed in conjunction with FIGS. 29, 30 and 25. Edit AR tutorial module 158 is configured to edit tutorial motion 216as disclosed in conjunction with FIG. 31 .

Record AR tutorial module 156 is configured to record the AR tutorialdata 202 for a presenter 236. For example, record AR tutorial module 156may generate AR tutorial data 202 including AR tutorial video 228.Record AR tutorial module 156 is configured to record additional ARtutorial video 228 in a recording 150 state and playback AR tutorialvideo 228 in a playing back 152 state.

Generate AR tutorial module 160 is configured to take the AR tutorialdata 202 and generate an AR tutorial module 246 that can be selected andrun by a user of the AR tutorial 143 or edited or recorded by apresenter of the AR tutorial 143. Generate AR tutorial module 160 isconfigured to generate AR tutorial data 202 as disclosed in conjunctionwith FIG. 2 . Generate AR tutorial module 160 generates the AR tutorialmodule under capture 832 of AR tutorial carousel 702 to include ARtutorial data 202 such as tutorial information 232 so that added effects1406 and added effects 1456 can be added to captured images 1414, 1464taken of users 238 of the AR tutorial 143 and so that AR effects 219 canbe added to captured images or live images of the user 238 such as isdisclosed in conjunction with FIG. 8 . In some embodiments, generate ARtutorial module 160 is separate from AR tutorial module 102.

Referring to FIG. 2 , AR tutorial data 202 is the data associated withAR tutorial 143. Step N 204 indicates data for a step of the AR tutorialdata 202. Step N 204 indicates a step associated with parameters, suchas the parameters disclosed in conjunction with FIGS. 2 and 3 . Eachstep, such as step 1 1608A, step 2 1608B, and step 3 1608C of FIG. 16has step N 204 associated with the step 1 1608A, step 2 1608B, and step3 1608C. Visible area 206 is the portion of the AR tutorial video 228,live images 134, or user video 230 to be displayed on the screen 132.Zoom level 208 indicates a zoom level of the AR tutorial video 228, liveimages 134, or user video 230 to be displayed on the screen 132.Playback speed value 210 indicates a playback speed, such as a playbackspeed value 210, of the AR tutorial video 228 or user video 230. Starttime 212 and end time 214 indicate a start time and an end time forparameters of step N 204. There may be multiple values for parametersfor step N 204, such as two different zoom levels 208, with each havinga start time 212 and end time 214. Parameters may have multiple starttimes 212 and end times 214 within step N 204 indicating differentvalues for the parameter, such as zoom level 208, visible area 206, andso forth. In some embodiments, start time 212 indicates a start timewithin the tutorial video 228 and/or user video 230 when an adjustmentapplies to the tutorial video 228 and/or user video 230. In someembodiments, adjustments, such as zoom level 208, is reset with thestart of a new step.

Tutorial motion 216 is data that indicates motion of the application ofa beauty product 224 as disclosed herein. AR motion 217 is data thatindicates motion to be applied to the live images 134 and/or user video230 as disclosed herein. Tutorial effects 218 is data that indicateschanges to the AR tutorial video 228 from the application of a beautyproduct 224 to the body part 222 of the presenter 236 as disclosedherein. AR effects 219 is data that indicates changes to make to theuser video 230 and/or live images 134 of the user 238 to simulate thetutorial effects 218 on the body part 222 of the user 238 in the uservideo 230 or live images 134 as disclosed herein.

Pause 220 is a time when the playing of the AR tutorial video 228 is tobe paused. Pause 220 may further indicate how the AR tutorial video 228is to return to playing or recording, such as by a set amount of timefor a pause or an indication by the user 238, such as a tap of a fingerto continue. Body part 222 is an indication of a body part of thepresenter 236 and/or user 238 to which the beauty product 224 isapplied. Customization 226 is customizations of step N 204 by the user238, such as a change in playback speed value 210 by the user 238 or achange in the color 316 or brush width 318 of a beauty product data 302.Beauty product 224 corresponds to a beauty product with associatedbeauty product data 302 as disclosed herein.

AR tutorial video 228 is the video of the presenter 236 for step N 204of the AR tutorial data 202. User video 230 is video of the user 238taking or playing the AR tutorial 143 of AR tutorial data 202. Uservideo 230 may be recorded for sending or storing by the user 238.Tutorial information 232 is information that is associated with the ARtutorial data 202, such as capture 832, screens 900 and 1000, preview ofAR tutorial 1102, shows associated with the AR tutorial data 202, addedeffects 1406, and so forth.

UI items 234 are UI items that may be used by AR tutorial 143. Anexample UI item is a label “Next Step Eye Liner” as a label for next1702 as illustrated in FIG. 17 . Presenter 236 corresponds to thepresenter of the AR tutorial data 202. Presenter 236 may includeinformation such as a presenter name, a website associated with thepresenter, images of the presenter, and so forth. User 238 correspondsto the user of the AR tutorial data 202. User 238 may includeinformation such as a username, a website associated with the user,images of the user, and so forth.

Options 240 may be options for the AR tutorial data 202, such as whetherthe user 238 may change any parameters of the AR tutorial data 202 suchas a playback speed value 210. Statistics 242 may be statistics that aregathered about AR tutorial data 202. Some example statistics include howmany users 238 select the capture 832 (FIG. 8 ) compared with how manyusers 238 view the capture 832; what percentage of users 238 completethe AR tutorial data 202; at which steps, do users 238 who do notcomplete the AR tutorial data 202 leave the tutorial; how often does auser 238 send, post, and/or save captured images; how many users 238return to use the AR tutorial 143 after leaving the AR tutorial 143;and/or other statistics.

Preview 244 is one or more previews of the AR tutorial data 202. Forexample, the images of screen 1000 of FIG. 10 , the images in ARtutorial display 142 in screen 1100 of FIG. 11 , images 909B of FIG. 9 ,and so forth. Additional examples of a preview 244 are disclosed herein.AR tutorial module 102 is code and data that may be executed orinterpreted by the mobile device 146 to preview, edit, generate, record,and/or play the AR tutorial data 202. AR tutorial icon 805 of ARtutorial carousel 702 provides a preview image under the capture 832 aswell as data and/or code so that the user 238 may preview the results ofthe AR tutorial 606, and AR tutorial 606 includes data and/or code sothat when invoked, e.g., selection of AR tutorial icon 805 under capture832, the AR tutorial 606 may be edited and/or played. In someembodiments, AR tutorial icon 805 may invoke an application that is areduced version of AR tutorial 606 that provides the AR effects 219 andother information related to the AR tutorial 606. In some embodiments,AR tutorial icon 805 may refer to data that another applicationassociated with AR tutorial carousel 702 uses to provide informationrelated to the AR tutorial 606 such as the AR effect 219 of FIG. 8 .

Referring to FIG. 3 , beauty product data 302 is the data associatedwith a beauty product 224. Beauty product information 304 indicatesinformation regarding the beauty product data 302, such as a name of thebeauty product, a website related to the beauty produce, a price of thebeauty product, and so forth. In some embodiments, beauty productinformation 304 includes information regarding effects that may beexpected to occur to a body part 308 in the application of the beautyproduct 224. For example, the effects may indicate that a body part witha particular skin tone may expect a color change as indicated.

Purchase page 306 is one or more websites or pages associated with thebeauty product 224. Body part 308 indicates a portion of a body of aperson that is associated with the beauty product 224. Example bodyparts include an eyelid, an eye, a nose, lips, and so forth. In someembodiments, body part 308 is retrieved from a database of beautyproducts where the body part 308 is associated with the beauty product224. For example, one or more of the data disclosed herein may beretrieved from beauty product data 302 of FIG. 3 .

Ingredient 310 is one or more ingredients of the beauty product 224.Ingredient percentage 312 indicates a percentage of an ingredient 310.Variations 314 indicates variations of the beauty product 224. Examplevariations 314 include color 316 and brush width 318. Variations 314 mayinclude a selection menu 320 that assists a user 238 in selectingvariations 314 of the beauty product data 302. Average rating 322indicates an average of ratings 324 of the beauty product 224. Ratings324 are ratings of the beauty product 224 by users 238, presenters 236,and/or other people. The ratings 324 may be displayed from websites.Images of beauty product 326 is one or more stored images of beautyproduct 224 that may be used for display purposes to the user 238, suchas beauty product 224A and beauty product 224B of FIG. 4 . Stored imagesof beauty products may be used for identifying the beauty product 224from images of the beauty product 224 such as during recording the ARtutorial 143. Competing products 328 are competing products that may bedisplayed to users 238 to switch the beauty product 224 or compare thebeauty product 224 with competing products 328. Product codes 330 may becodes and/or other information to assist AR tutorial module 102 inidentifying a beauty product 224 from live images 134 of the beautyproduct. Association data 332 is data that associates the beauty productdata 302 with a beauty product 224, AR tutorial data 202, and-or an ARtutorial 143. Effects 334 is an indication of the effects of applyingthe beauty product 224 with beauty product data 302 to a person. Forexample, effects 334 may be one or more of the following: a color, atexture, a transparency, a width, a length, a suggested skin type forthe body part, and a suggested hair type for the body part.

In some embodiments, AR tutorial module 102 is configured to displayinformation related to the beauty product 224 in response to a selectionof the beauty product 224 within the AR tutorial 143. For example, ARtutorial module 102 displays screen 1500 with information related tobeauty product 224B in response to a selection of beauty product 224B byuser 238.

FIG. 4 illustrates the operation 400 of an AR tutorial 143, inaccordance with some embodiments. The camera 130, screen 132, ARtutorial display 142, AR tutorial 143, AR user display 144, and mobiledevice 146 are the same or similar as the elements with like numbers ofFIG. 1 . Playback speed value 210, tutorial motion 216, AR motion 217,tutorial effects 218, AR effects 219, body part 222A, body part 222B,beauty product 224A, 224B, and, user 238 are the same or similar as theelements with like numbers of FIG. 2 . In some embodiments the AR userdisplay 144 is a live image of a user 238 viewing or taking the ARtutorial 143.

Presenter 236 comprises an image of the presenter of the AR tutorial143. For example, the presenter 236 in FIG. 4 is an image of a personthat is presenting the AR tutorial 143 on how to apply make-up. Bodyparts 222A and body part 222B are the same or similar as body part 222.Beauty product 224A and beauty product 224B are each an image of beautyproduct 326 as disclosed in conjunction with FIG. 3 . Beauty products224A and beauty product 224B have associated beauty product data 302.Beauty product 224A and beauty product 224B each correspond to a beautyproduct 224 of AR tutorial data 202. In some embodiments, a different UIitem 234 is used to indicate the beauty products 224A and beauty product224B. For example, the beauty products 224 may be displayed in the ARuser display 144. In another example, the beauty products 224 may not bedisplayed. In some embodiments, beauty products 224 may be displayed inresponse to a scroll left, scroll right, scroll up, and/or scroll downindication by the user 238. In some embodiments, beauty products 224 mayinclude an animated display. In some embodiments, beauty products 224may include an advertisement that plays within screen 132.

Playback speed value 210 indicates a value of the playback speed. Theplayback speed in FIG. 4 is illustrated as 1×, which represents 1 timesa normal playback speed. Playback speed bar 414 indicates a playbackspeed of the AR tutorial 143. In some embodiments, a user 238 mayinteract with the playback speed bar 414 to change the playback speedvalue 210. In some embodiments, the playback speed value 210 and/orplayback speed bar 414 are not displayed. In some embodiments, adifferent user interface element is used to indicate the playback speedvalue 210 and/or the playback speed bar 414.

AR motion 217 is the motion extracted from the tutorial motion 408 ofthe presenter 236, such as via determine motion module 122 as disclosedin conjunction with FIG. 1 . Determine motion module 122 extracts thetutorial motion 216 of FIG. 2 , in accordance with some embodiments. ARmotion 217 is the same or similar as tutorial motion 216 of FIG. 2 , inaccordance with some embodiments.

User 238 is the person using the mobile device 146 to view, play, ortake the AR tutorial 143. In some embodiments, an image of the user 238is displayed in the AR user display 144. In some embodiments, the user238 indicates the image of a user in the AR user display 144. In someembodiments, the user 238 is a person who is the user of the mobiledevice 146 and viewing, playing, or taking the AR tutorial 143. In someportions of the disclosure, the user 238 is referred to as a personhere, which can include any gender. In some portions of this disclosure,the user 238 is referred to as a woman; however, the user 238 may be aperson of any gender.

AR effects 219 are determined from the tutorial effects 218 that areextracted from AR tutorial 143. For example, as disclosed in conjunctionwith FIG. 1 , determine effects module 128 extracts tutorial effects 218of FIG. 2 from AR tutorial video 228. In some embodiments, AR effects219 are determined by the determine effects module 128. In someembodiments, AR effects 219 may be displayed differently. For example,AR effects 219 may be displayed separately from the user 238. In someembodiments, AR effects 219 may be adjusted to either reduce orintensify tutorial effects 218 on the user 238. For example, anintensity bar may be displayed on screen 132 where the user 238 mayadjust AR effects 219 to either reduce or intensity the application ofthe tutorial effects 218 as the AR effects 219 on the user 238.

Play bar 422 is a bar indicating information regarding the playing ofthe AR tutorial 143. Current play 424 indicates the current position ofplay within the AR tutorial 143. Step marks 426A, 426B indicate the endand start of steps. Time in tutorial 430 indicates the current time ofthe current play 424. In some embodiments, time in tutorial 430 is atime left in the AR tutorial 143. In some embodiments, time in tutorial430 indicates the time from the start of the AR tutorial 143. Theinformation regarding playing of the AR tutorial 143 illustrated in FIG.4 is just one example. In example embodiments, the information forplaying the AR tutorial 143 may be presented differently, one or morepieces of information may not be presented, or additional informationmay be presented that is not illustrated in the example in FIG. 4 .

Back 428 is a user interface element, such as an icon, that will takethe AR tutorial 143 back one or more steps. For example, selecting back428 may take the user 238 out of the AR tutorial 143. Back 428 is acommand 112 as disclosed in conjunction with FIG. 1 .

Capture 432 is a user interface element, such as an icon, that whenselected captures an image of the AR user display 144 and/or AR tutorialdisplay 142. In some embodiments, a long press on capture 432 captures avideo of the AR user display 144 and/or AR tutorial display 142. Capture432 is a command 112 as disclosed in conjunction with FIG. 1 . Thefunctionality of capture 432 may be invoked in different ways asdisclosed in conjunction with FIG. 1 . The functionality of back 428and/or capture 432 may depend on the context with which they areselected. Synchronized 434 indicates that tutorial motion 216 andtutorial effects 218 are synchronized with AR motion 217 and AR effects219, respectively, in accordance with some embodiments. For example, thesynchronize effects and motion module 106, the generate user imagesmodule 108, and/or the generate tutorial images module 104 may performthe functionality of synchronized 434 as disclosed herein.

The AR user display 144 is generated by the generate user images module108. The generate user images module 108 may perform image adjustmentsfrom the live images 134 and/or user video 230. For example, thegenerate user images module 108 may take an image of the user 238captured via the camera 130 and locate a body part 222 associated withstep N 204 of the AR tutorial 143. For example, as illustrated in FIG. 5a right eye. Generate user images module 108 may then zoom in on theimage captured of the user 238 so that the body part 222 is proportionalin size to the body part 222 of the presenter 236. The user 238 may bemoving so that generate user images module 108 may continually adjustthe display of the image of the user 238 on the AR user display 144. Insome embodiments, a creator of the AR tutorial 143 may determine avisible area 206 (FIG. 2 ), zoom level 208, and/or playback speed value210 for the AR tutorial 143 that can be used by the generate user imagesmodule 108 to perform image adjustments to the image of the user 238 fordisplay in the AR user display 144.

In some embodiments, AR tutorial 143 will loop for each step of the ARtutorial 143 until the user 238 selects to move to a next step. This isdescribed in further detail in conjunction with FIGS. 15 and 16 . Insome embodiments, AR tutorial 143 progresses to a next step, such aspast the step marks 426A and 426B, without looping or pausing. In someembodiments, AR tutorial 143 pauses at the end of a step such as step 11608A, step 2 1608B, or step 3 1608C of FIG. 16 . In some embodiments,AR tutorial 143 pauses at the end of a step until user 238 indicates tomove to the next step. For example, the user can indicate a command tomove to the next step by a selection of a user interface element, amovement of the user 238 within the AR user display 144, a sound, and soforth. The user 238 can perform a customization 226 of FIG. 2 , such aschanging a color 316 and/or brush width 318.

The AR tutorial module 102 presents a selection menu 320 to the user238, and, in response to a selection of a customization 226 by the user238, updates AR effects 219 and/or AR motion 217, and displays the user238 in the AR user display 144 based on the updated AR effects 219and/or AR motion 217. Screen 1500 of FIG. 15 illustrates an examplewhere a different current color 1508 may be selected by the user 238.

The user 238 can switch between a full-screen AR user display 144 and ahalf-screen or partial screen 132 view of the user 238 with the AReffects 219 and/or AR motion 217. The user 238 is presented with anoption to turn off AR effects 219. The user 238 may then apply a beautyproduct data 302 to themselves and use the AR motion 217 to assistthemselves in applying the beauty product data 302, in accordance withsome embodiments.

FIGS. 5A, 5B, and 5C illustrate the operation of an AR tutorial 143, inaccordance with some embodiments. The camera 130, screen 132, ARtutorial display 142, AR tutorial 143, AR user display 144, and mobiledevice 146 are the same or similar as the elements with like numbers ofFIG. 1 . Tutorial motion 216A, 216B, 216C, tutorial effects 218A, 218B,218C, AR motion 217A, 217B, 217C, AR effects 219A, 219B, 219C, presenter236, beauty products 224, playback speed value 210, and user 238 are thesame or similar as the elements with like numbers of FIG. 2 . Playbackspeed bar 414, play bar 422, current play 524A, current play 524B,current play 524C, step marks 426, back 428, capture 432, synchronized434 are the same or similar as the elements with like numbers of FIG. 4.

FIGS. 5A, 5B, and 5C illustrate that tutorial motion 216A, 216B, 216Cand tutorial effects 218A, 218B, 218C are synchronized 434 with ARmotion 217A, 217B, 217C, and AR effects 219A, 219B, 219C, respectively.FIG. 5A illustrates a time in tutorial 530A of “0:12” or 12 seconds.FIG. 5B illustrates a time in tutorial 530B of “0:17” or 17 seconds.FIG. 5C illustrates a time in tutorial 530C of “0:22” or twenty-twoseconds. The AR motion 217A, 217B, and 217C simulate the motion oftutorial motion 216A, 216B, 216C, respectively. Tutorial motion 216A,216B, 216C refer to the changes in the movement of the beauty product224A, in accordance with some embodiments. The AR effects 219A, 219B,219C simulate the tutorial effects 218A, 218B, 218C, respectively.Synchronizing the tutorial motion 216 and tutorial effects 218 with theAR motion 217 and AR effects 219, respectively, enables the user 238 tosee how the application of the beauty products 224 would look on her ina similar way as the application of the beauty products 224 are appliedto the presenter 236. The AR tutorial depicts the presenter 236 applyinga beauty product to a body part of the presenter 236.

FIGS. 6 and 7 are disclosed in conjunction with one another. FIG. 6illustrates a flow diagram for using an AR tutorial 606, in accordancewith some embodiments. FIG. 7 illustrates a point of entry 608, inaccordance with some embodiments. Screen 600 is the same or similar asscreen 132 of FIG. 1 . Camera 130, AR tutorial display 142, AR userdisplay 144, mobile device 146 are the same or similar as the elementswith like numbers of FIG. 1 . AR tutorial 606 is the same or similar asAR tutorial 143 of FIG. 1 .

A user 238 (not illustrated) of AR tutorial 606 uses a mobile device 146with a camera 130. The user 238 selects 610 a command 112 to enter theAR tutorial 606 from a point of entry 608. For example, as illustratedin FIG. 8 , a user 238 selects 610 command 112 capture 832 to enter ARtutorial 606 where the AR tutorial 606 is being highlighted or previewedin screen 800. As illustrated in FIG. 7 , a user may enter the ARtutorial 606 via an AR tutorial carousel 702, discover show 704, context706, or search 708. Context 706 indicates that AR tutorials 606 (FIG. 6) may be offered to the user 238 within the context of another task theuser 238 is performing. For example, the user may be watching a video orreceive an image and be offered a link to AR tutorial 606 that willteach the user how to apply makeup to look like a person in the video orimage.

FIGS. 8, 9, and 10 illustrate points of entry, in accordance with someembodiments. FIG. 8 illustrates an AR tutorial carousel 702, inaccordance with some embodiments. Screen 800 is the same or similar asscreen 132 of FIG. 1 . Camera 130, AR user display 144, and mobiledevice 146 are the same or similar as the elements with like numbers ofFIG. 1 . User 238 is the same or similar as user 238 of FIG. 2 . The AReffects 219, and back 428 are the same or similar as the elements withlike numbers of FIG. 4 . AR tutorial 606 is the same or similar as ARtutorial 606 of FIG. 6 . Finger 810 is a finger of the user 238.

Capture 832 indicates that the AR tutorial 606 highlighted by capture832 is being previewed on the AR user display 144 with the user 238having AR effects 219 from the AR tutorial 606 added to the user 238.The user 238 can preview AR tutorial 606 and determine whether to enterAR tutorial 606. If user 238 selects 610 (FIG. 6 ) capture 832, such aswith finger 810, then the user 238 enters the AR tutorial 606 (FIG. 6 ).For example, the user can view the AR tutorial 606 via screen 600 ofFIG. 6 . Selecting capture 832 is a command 112 of FIG. 1 . The user 238may preview other AR tutorials that are indicated as part of the ARtutorial carousel 702. For example, the user can swipe a finger 810 tomove different AR tutorials under capture 832. AR tutorial carousel 702is a collection of items that can be put under capture 832 and thenselected by the user 238. Selects 610 (FIG. 6 ) may be performed in adifferent way. Other AR tutorials may be moved under the capture 832element in different ways. In some embodiments, a selection of back 428would return the user 238 to a screen (not illustrated) before screen800 of FIG. 8 .

FIG. 9 illustrates points of entry 608 into AR tutorial, in accordancewith some embodiments. Screen 900 is the same or similar as screen 132of FIG. 1 . Mobile device 146 is the same or similar as the elementswith like numbers of FIG. 1 . Back 428 is the same or similar as theelement with like number of FIG. 4 . Finger 810 is the same or similaras the element with like number of FIG. 8 . AR tutorials 906A and 906Bare the same or similar as AR tutorial 606 of FIG. 6 . Discover shows904 is the same, or similar as discover shows 704 of FIG. 7 . Search 902is the same or similar as search 708 of FIG. 7 .

A select 610 (FIG. 6 ) by user 238 (not illustrated), such as withfinger 810, brings the user 238 into screen 600 of FIG. 6 . The user 238may select 610 an AR tutorial 906A, 906B by entering one or more searchterms in search 902 and selecting an AR tutorial from search results(not illustrated). The user may select 610 (FIG. 6 ) an AR tutorial 906Afrom subscriptions 906. The user may select 610 AR tutorial 906B fromdiscovery shows 904. AR tutorials 906A, 906B may be displayed withinformation 907B and/or image 909B. Information 907B and/or image 909Bmay include information from AR tutorial data 202 such as tutorialinformation 232, presenter 236, and so forth. In some embodiments, aselection of back 428 returns the user to a screen (not illustrated)before screen 900 of FIG. 9 .

FIG. 10 illustrates a point of entry 608, in accordance with someembodiments. Screen 1000 is the same or similar as screen 132 of FIG. 1. AR user display 144 and mobile device 146 are the same or similar asthe elements with like numbers of FIG. 1 . Back 428 is the same orsimilar as the element with like number of FIG. 4 . AR tutorial 606 isthe same or similar as AR tutorial 606 of FIG. 6 . Finger 810 is thesame or similar as the finger 810 of FIG. 8 . Select icon 1002illustrates an icon that when selected by the user, brings the user toscreen 600 of FIG. 6 , in accordance with some embodiments. Screen 1000illustrates a preview of AR tutorial 606, in accordance with someembodiments.

FIG. 11 illustrates a point of entry, in accordance with someembodiments. Mobile device 146, AR tutorial display 142, and AR userdisplay 144 are the same or similar as the elements with like numbers ofFIG. 1 . User 238 is the same or similar as user 238 of FIG. 2 . AReffects 219, capture 432, back 428 are the same or similar as theelements with like numbers of FIG. 4 . Finger 810 is the same or similaras the element with like number of FIG. 8 .

Preview of AR tutorial 1102 may include educational information thateducates the user 238 on how to use the AR tutorial 606 that is thesubject of the preview of AR tutorial 1102, such as indications of howto step through AR tutorial 606. AR effects 219 are added to the user238. User 238 may preview the results of AR tutorial that is the subjectof AR tutorial display 142 with the AR effects 219 by viewing screen1100.

A select 610 (FIG. 6 ) of capture 432, such as with finger 810, bringsthe user 238 to screen 600 of FIG. 6 . The user 238 is already in ARtutorial 606 of FIG. 6 and the preview of AR tutorial 1102 is a startscreen for AR tutorial 606 to provide information to the user 238, suchas educational information on how to use AR tutorial 606, types ofmake-up used, and so forth. In some embodiments, additional informationmay be displayed in screen 1100 such as information from AR tutorialdata 202.

Returning to FIG. 6 , once a user selects 610 to enter the AR tutorial606 from a point of entry 608 as disclosed in conjunction with FIGS.7-11 , the user's options include capture image 616A, completing the ARtutorial 622, or exit 613 the AR tutorial 606. The user selection ofcapture image 616A brings the user to screen 1200B or 1300. The user maybe done 612A with preview and send flow 628A and return to screen 600 asdisclosed in conjunction with FIGS. 12 and 13 . The user may move frompreview and send flow 628A to full camera 630 via switch to/from fullcamera 632. For example, as illustrated in FIG. 12 , the user 238 mayselect full screen 1216 to enter screen 1300 of FIG. 13 via full screen1302.

In accordance with some embodiments, the user completing AR tutorial 606brings the user to full camera 630. For example, as illustrated in FIG.13 , AR user display 144 is in a full screen mode. The user may return620 to screen 600 as disclosed in conjunction with FIG. 13 . Forexample, as illustrated in FIG. 13 , the user 238 may select back 428.

The user may be done with full camera 624. For example, as illustratedin FIG. 13 , user 238 may select back 428 from screen 1300 to be donewith full camera 624 and be brought to return to point of entry 614. Theuser may switch to/from full camera 632 to go back to preview and sendflow 628A. For example, as illustrated in FIG. 13 , user 238 may selectback 428 to be returned to screen 1200A of FIG. 12 when the user 238entered 1300 via full screen 1302. The user may capture image 616B to bebrought to preview and send flow 628B. For example, as illustrated inFIG. 13 user 238 may capture an image by selecting capture 432 to bebrought to screen 1400 of FIG. 14 . The user may be done 612B frompreview and send flow 628B. For example, as disclosed in conjunctionwith FIG. 14 , user 238 may return 1407 to return to point of entry 614.

The user may exit 613 from AR tutorial 606 to enter return to point ofentry 614. For example, user 238 of FIG. 4 may select back 428 to returnto a previous screen. Return to point of entry 614 returns the user toanother screen that is not part of the AR tutorial 606 and may be thelast screen the user was presented before entering the AR tutorial 606such as screen 800 of FIG. 8 .

FIG. 12 illustrates capturing an image, in accordance with someembodiments. Screen 1200A and screen 1200B are the same or similar asscreen 132 of FIG. 1 . Camera 130, AR tutorial display 142, AR userdisplay 144, and mobile device 146 are the same or similar as theelements with like numbers of FIG. 1 . Presenter 236 and user 238 arethe same or similar as the elements with like numbers of FIG. 2 .Tutorial motion 408, tutorial effects 218, AR motion 217, AR effects219, back 428, capture 432 are the same or similar as the elements withlike numbers of FIG. 4 . AR tutorial 606 is the same or similar as ARtutorial 606 of FIG. 6 . Finger 810 is the same or similar as finger 810of FIG. 8 .

The user 238 may select capture 432 with her finger 810 during ARtutorial 606. When the user 238 selects capture 432, it invokes captureimage 1202 command, which captures captured image 1214. Capture image1202 is a command 112. The user 238 may edit and enhance 1208 thecaptured image 1214, save 1210 the captured image 1214, send 1212 thecaptured image 1214, or back 1204, 428, in accordance with someembodiments. The AR effects 219 are included in the captured image 1214and the AR motion 217 is not included in the captured image 1214, inaccordance with some embodiments. In some embodiments, the AR motion 217is included in the captured image 1214. In some embodiments, an optionis provided to the user 238 to include the AR motion 217.

Return 1206 returns the user from screen 1200B to screen 1200A, inaccordance with some embodiments. Screen 1200B may return 1206 to screen1200A after the user 238 saves 1210 the captured image 1214, sends 1212the captured image 1214, or selects back 1204, 428, in accordance withsome embodiments. In some embodiments, when the user 238 selects capture432, such as with finger 810, AR user display 144 occupies the entirescreen 1200B such as screen 1300 rather than half the screen 1200B. Insome embodiments, selecting full screen 1216 brings the user 238 toscreen 1300 of FIG. 13 .

FIG. 13 illustrates capturing an image, in accordance with someembodiments. Screen 1300 is the same or similar as screen 132 of FIG. 1. Camera 130, screen 1300, mobile device 146, AR user display 144 arethe same or similar as the elements with like numbers of FIG. 1 . User238, AR effects 219, back 428, 424 capture 432 are the same or similaras the elements with like numbers of FIG. 4 . AR tutorial 606 is thesame or similar as AR tutorial 606 of FIG. 6 . Finger 810 is the same orsimilar as finger 810 of FIG. 8 .

Screen 1300 is a live image of the user 238 from camera 130. The user238 enters screen 1300 by complete AR tutorial 622 of FIG. 6 or byselecting full screen 1216 of FIG. 12 to invoke full screen 1302command, in accordance with some embodiments. Full screen 1302 is acommand 112. The user 238 enters screen 1400 of FIG. 14 by selectingcapture 432, which invokes the select capture 1304 command. For example,with her finger 810. Edit and enhance 1308 provides options for addingto the screen 1300 added effects 1306. The AR effects 219 are includedin screen 1300. AR user display 144 is displayed on a full screen. Back428 takes the user 238 to a previous screen such as screen 1200A of FIG.12 or screen 132 of FIG. 4 , in accordance with some embodiments.

FIG. 14 illustrates captured images 1414 and 1464, in accordance withsome embodiments. Screens 1400, 1450 are the same or similar as screen132 of FIG. 1 . Camera 130, AR user display 144, and mobile device 146are the same or similar as the elements with like numbers of FIG. 1 . AReffects 219, back 428, and user 238 are the same or similar as theelements with like numbers of FIG. 4 . AR tutorial 606 is the same orsimilar as AR tutorial 606 of FIG. 6 . Finger 810 is the same or similaras finger 810 of FIG. 8 .

The AR user display 144 is displayed as a full screen on screens 1400,1450. The user 238 of FIG. 14 comes to screen 1400 after selecting acommand to capture 1410 an image. For example, user 238 of FIG. 13selects capture 432 to arrive at screen 1400. Capture 1410, selects send1401, return 1407, and return 1457 are commands 112. The user 238 maysend 1212 the captured image 1414. For example, user 238 may select send1212, which sends the captured image 1414. AR tutorial 606 may invokereturn 1407, 1457 after the captured image 1414 is sent. The user 238may return 1407, 1457 to a previous screen such as screen 1300, byselecting back 1404, back 1454, or back 428. The user 238 may perform aselection with her finger 810. For example, user 238 may select send1212 with a finger 810. The user 238 may save 1410 the captured image1414. The user 238 may edit and enhance 1408 the captured image 1414such as modify the image to add added effects 1406. Captured image 1414includes AR effects 219.

The user 238 comes to screen 1450 after selecting a command to send animage of screen 1400. For example, the user 238 selects send 1210, whichinvokes selects send 1401 command and displays screen 1450. AR tutorial606 may modify the captured image 1464 to add added effects 1456 beforesending the captured image 1464. The added effects 1456 identify thecreator of the AR tutorial 606, identify shows associated with the ARtutorial 606, and/or provide other information from AR tutorial data202. As illustrated, “Face” may be the originator of the AR tutorial 606and the AR tutorial 606 may have a name of “Celebrity makeover.” “MetGala Moonshine” may be a related AR tutorial. The user 238 of FIG. 14may be named “JUSTINE” as illustrated on screen 1450. The user 238 mayreturn 1457 either based on a timer or selection of a back 428, 1454.The added effects 1456 may be included in AR tutorial data 202 such astutorial information 232 or presenter 236.

FIG. 15 illustrates information for a beauty product, in accordance withsome embodiments. Mobile device 146 and AR tutorial 143 are the same orsimilar as the elements with like numbers of FIG. 1 . Screen 1500 is thesame or similar as screen 132 of FIG. 1 . Back 428 and beauty product224B are the same or similar as elements with like numbers of FIG. 4 .

A user may arrive at screen 1500 from select 1510. Select 1510 is acommand 112. For example, referring to FIG. 4 , a user 238 may selectbeauty product 224B. Back 428 will bring the user back to the screenwhere the user came from. For example, back to screen 132 of FIG. 4 .Manufacturer 1502 is a manufacturer of the beauty product 224B.Manufacturer may be included in beauty product data 302 of FIG. 3 .Hotlink 1504 is one or more hotlinks to purchase or provide additionalinformation regarding beauty product 224B. Hotlink is included inpurchase page 306 of FIG. 3 and/or beauty product information 304.Current color 1508 indicates a current selection of a variation of thebeauty product 224B, such as color 316 of FIG. 3 . Competitors 1506provides an indication of competitors of the beauty product 224B.Competitors may be included in competing products 328. A selection ofcompetitors 1506 brings the user to a screen (not illustrated) withinformation regarding the competitor. Different information may bedisplayed than is illustrated such as information from AR tutorial data202 of FIG. 2 and/or beauty product data 302 of FIG. 3 .

FIG. 16 illustrates steps of an AR tutorial 1600, in accordance withsome embodiments. Step mark 1612A and step mark 1612B are the same orsimilar as step mark 426A and step mark 426B of FIG. 4 . Current play1614 is the same or similar as current play 424. Play bar 1610 is thesame or similar as play bar 422. Beauty products are the same or similaras beauty product data 302 as disclosed in conjunction with FIG. 3 .Step 1 1608A, step 2 1608B, and step 3 1608C are the same or similar asstep N 204 of FIG. 2 . AR tutorial 1600 is the same or similar as ARtutorial 143.

The play bar 1610 indicates a duration of time of the AR tutorial 1600.Step 1 1608A is from the beginning of the play bar 1610 to step mark1612A. Step 2 1608B is from step mark 1612A to step mark 1612B. Step 31608C is from step mark 1612B to end of play bar 1610. Step 1 1608A,step 2 1608B, and step 3 1608C may include different parameters such asdifferent beauty products for step 1 1608A, step 2 1608B, and step 31608C. Step 1 1608A, step 2 1608B, and step 3 1608C may have one or morebeauty products. For example, as illustrated in FIG. 17 , screen 1700A,which may be step 1 1608A of AR tutorial 1600, includes beauty product224A and beauty product 224B; and, screen 1700B, which may be step 21608B, includes beauty product 1706A and beauty product 1706B. Time intutorial 1630 indicates “0:22” or twenty-two seconds.

Each step, such as step 1 1608A, step 2 1608B, and step 3 1608C, mayindicate a different step in a tutorial. In one example, step 1 1608A isa cleanse with a beauty product 1606A, step 2 1608B is applying afoundation with a beauty product 1606B, and step 3 1608C is applying ablush with a beauty product 1606C.

FIG. 17 illustrates forward 1605 and back 1603 of FIG. 16 , inaccordance with some embodiments. Camera 130, screen 132, AR tutorial143, AR user display 144, and mobile device 146 are the same or similaras the elements with like numbers of FIG. 1 . Presenter 236, beautyproducts 224, playback speed value 210, playback speed bar 414, user238, play bar 422, step marks 426, back 428, capture 432, tutorialmotion 216, tutorial effects 218, AR motion 217, AR effects 219, currentplay 424A, and current play 424B, are the same or similar as theelements with like numbers of FIG. 4 . Forward 1605 is the same orsimilar as forward 1604A, forward 1604B, and forward 1604C of FIG. 16 .Back 1603 is the same or similar as back 1602A, back 1602B, and back1602C of FIG. 16 . AR tutorial 1600 is the same or similar as ARtutorial 1600 of FIG. 16 . Time in tutorial 1730A indicates “0:22” ortwenty-two seconds. Time in tutorial 1730B indicates “0:27” or 27seconds.

As illustrated in FIG. 17 , the screen 1700A indicates that step 1, suchas step 1 1608A, is over. The user 238 may invoke forward 1604 byselecting next 1702, in accordance with some embodiments. Forward 1604is a command 112, in accordance with some embodiments. The user 238 mayinvoke forward 1604 by moving the current play 424A after step mark426A, in accordance with some embodiments. The user 238 may invoke back1602A, back 1602B, or 1602C by selecting back 426, in accordance withsome embodiments. AR tutorial 1600 may respond to back 1602A, back1602B, or 1602C by moving the current play 424A to a position within aprevious step 1 1608A, step 2 1608B, step 3 1608C. Back 1602A, back1602B, and 1602C is a command 112. The user 238 may invoke back 1602A,back 1602B, or 1602C by moving current play 424B to a position beforestep mark 426A so that current play 424B is within a previous step. Theuser 238 may move among the step 1 1608A, step 2 1608B, and step 3 1608Cin a different way.

FIG. 18 is a block diagram showing an example of an AR tutorial system1800, in accordance with some embodiments. The AR tutorial system 1800includes client device 1802A and client device 1802B, which hosts anumber of applications including an AR tutorial client application 1804Aor AR tutorial client application 1804B, respectively. Client device1802A and client device 1802B include AR tutorial client application1804A and AR tutorial client application 1804B, respectively, which iscommunicatively coupled to an AR tutorial server system 1808 via anetwork 1806 (e.g., the Internet).

AR tutorial client device 1802A and AR tutorial client device 1802B areable to communicate and exchange data with the AR tutorial server system1808 via the network 1806. The data exchanged between AR tutorial clientapplication 1804A or AR tutorial client application 1804B and ARtutorial server system 1808, includes functions (e.g., commands toinvoke functions) as well as payload data (e.g., text, audio, video orother multimedia data). For example, AR tutorial client application1804A or AR tutorial client application 1804B may be the same or similaras AR tutorial module 102 and the data exchanged may include AR tutorialserver application 1814. AR tutorial server system 1808 may include ARtutorial data 202, beauty product date 302, the AR tutorial clientapplication 1804A, or AR tutorial client application 1804B, which may bedownloaded to the client device 1802A or client device 1802B. ARtutorial client application 1804A, AR tutorial client application 1804B,and-or AR tutorial server application 1814 may be the same or similar asAR tutorial module 102.

The AR tutorial server system 1808 provides server-side functionalityvia the network 1806 to AR tutorial client application 1804A or ARtutorial client application 1804B. While certain functions of the ARtutorial system 1800 are described herein as being performed by ARtutorial client application 1804A, AR tutorial client application 1804B,or by the AR tutorial server system 1808, the location of certainfunctionality either within the AR tutorial client application 1804A, ARtutorial client application 1804B, or the AR tutorial server system 1808is a design choice. For example, it may be technically preferable toinitially deploy certain technology and functionality within the ARtutorial server system 1808, but to later migrate this technology andfunctionality to the AR tutorial client application 1804A or AR tutorialclient application 1804B where a client device 1802A or client device1802B has a sufficient processing capacity. In some embodiments, ARtutorial server application 1814 provides functionality to the ARtutorial client application 1804A or AR tutorial client application1804B with regards to AR tutorials.

The AR tutorial server system 1808 supports various services andoperations that are provided to the AR tutorial client application 1804Aand AR tutorial client application 1804B. Such operations includetransmitting data to, receiving data from, and processing data generatedby the AR tutorial client application 1804A and AR tutorial clientapplication 1804B. This data may include, AR tutorial module 102, ARtutorial data 202, beauty product data 302, client device information,geolocation information, media annotation and overlays, social networkinformation, and live event information, as examples. Data exchangeswithin the AR tutorial system 1800 are invoked and controlled throughfunctions available via user interfaces (UIs) of the AR tutorial clientapplication 1804A and AR tutorial client application 1804B.

Turning now specifically to the AR tutorial system 1800, an ApplicationProgram Interface (API) server 1810 is coupled to, and provides aprogrammatic interface to, an application server 1812. The applicationserver 1812 is communicatively coupled to a database server 1818, whichfacilitates access to a database 1820 in which is stored data associatedwith AR tutorial client application 1804A, AR tutorial clientapplication 1804B, and AR tutorial server application 1814.

API server 1810 receives and transmits data between client device 1802Aor client device 1802B and application server 1812. Specifically, theAPI server 1810 provides a set of interfaces (e.g., routines andprotocols) that can be called or queried by the AR tutorial clientapplication 1804A and AR tutorial client application 1804B in order toinvoke functionality of the application server 1812. The API server 1810exposes various functions supported by the application server 1812,including account registration, login functionality, thesending/receiving of data and applications, via the application server1812, to-from AR tutorial client application 1804A or AR tutorial clientapplication 1804B. The data and applications may include AR tutorialmodule 102, AR tutorial data 202, beauty product data 302, media files(e.g., AR tutorials or advertisements for AR tutorials). The data may besent from client device 1802A to client device 1802B via AR tutorialserver system 1808. For example, client device 1802A may create ARtutorial data 202 and client device 1802B may download the AR tutorialdata 202 and execute AR tutorial module 102 with AR tutorial data 202from client device 1802B. Additionally, client device 1802A or clientdevice 1802B may access the Internet via AR tutorial server system 1808,in accordance with some embodiments. AR tutorial server system 1808provides access to client device 1802A or client device 1802B other dataincluding a collection of media data (e.g., tutorial or story), theretrieval of a list of friends of a user of a client device 1802A orclient device 1802B, the retrieval of such collections, the retrieval ofmessages and content, the adding and deletion of friends to a socialgraph, the location of friends within a social graph, opening anapplication event (e.g., relating to AR tutorial client application1804A or AR tutorial client application 1804B), and searching for ARtutorials.

The application server 1812 hosts a number of applications andsubsystems, including an AR tutorial server application 1814, an imageprocessing system 1816, and a social network system 1822. The ARtutorial server application 1814 implements a number of AR tutorialprocessing technologies and functions, particularly related to theadvertising for AR tutorials, creation of AR tutorials, and playing ofAR tutorials. As disclosed herein, text and media content from multiplesources may be aggregated into collections of content (e.g., calledstories, galleries, or AR tutorials). The application server 1812 alsoincludes an image processing system 1816 that is dedicated to performingvarious image processing operations, typically with respect to images orvideo received within the payload of a message at the AR tutorial clientapplication 1804A or AR tutorial client application 1804B. For example,in some embodiments, image processing system 1816 may perform one ormore of the functions of determine body part module 124, determineeffects module 128, or determine motion module 122.

The Application server 1812 supports various social networking functionsand services, and makes these functions and services available to the ARtutorial server application 1814. To this end, the social network system1822 maintains and accesses an entity graph 2004 (as shown in FIG. 20 )within the database 1820. Examples of functions and services supportedby the social network system 1822 include the identification of otherusers of the AR tutorial system 1800 with which a particular user hasrelationships or is “following”, and also the identification of otherentities and interests of a particular user, e.g., users that haveprovided AR tutorials. The application server 1812 is communicativelycoupled to a database server 1818, which facilitates access to adatabase 1820 in which is stored data associated with AR tutorialsprocessed by the AR tutorial server application 1814.

FIG. 19 is block diagram 1900 illustrating further details regarding theAR tutorial system 1800, according to example embodiments. The clientdevice 1802A or client device 1802B may include a messaging clientapplication 1905, which may include an ephemeral timer system 1902, acollection management system 1904 and an annotation system 1906. Forexample, the messaging client application 1905 may be used to edit,enhance, and/or send the screen of client device 1802A or client device1802B as a message or save as a story.

The application server 1812 may include a messaging client application1905. The ephemeral timer system 1902 is responsible for enforcingtemporary access to content permitted by messaging client application1905 and AR tutorial client application 1804A and AR tutorial clientapplication 1804B. To this end, the ephemeral timer system 1902incorporates a number of timers that, based on duration and displayparameters associated with a message, or collection of messages (e.g., astory), selectively display and enable access to messages and associatedcontent via the messaging client application 1905.

The collection management system 1904 is responsible for managingcollections of media (e.g., collections of text, image video, ARtutorials data, beauty products, and audio data). In some examples, acollection of content (e.g., messages, including images, video, text andaudio) may be organized into an “event gallery” or an “event story”and/or as a “Make Over Tutorial.” Such a collection may be madeavailable for a specified time period, such as the duration of an eventto which the content relates. For example, content relating to a musicconcert may be made available as a “story” for the duration of thatmusic concert. The collection management system 1904 may also beresponsible for publishing an icon that provides notification of theexistence of a particular collection to the user interface of the ARtutorial client application 1804A or AR tutorial client application1804B.

The collection management system 1904 furthermore includes a curationinterface 1908 that allows a collection manager to manage and curate aparticular collection of content. For example, the curation interface1908 enables an event organizer to curate a collection of contentrelating to a specific event (e.g., delete inappropriate content orredundant messages). Additionally, the collection management system 1904employs machine vision (or image recognition technology) and contentrules to automatically curate a content collection. In certainembodiments, compensation may be paid to a user for inclusion ofuser-generated content into a collection. In such cases, the curationinterface 1908 operates to automatically make payments to such users forthe use of their content.

The annotation system 1906 provides various functions that enable a userto annotate or otherwise modify or edit media content associated with amessage. For example, the annotation system 1906 provides functionsrelated to the generation and publishing of media overlays for messagesprocessed by the AR tutorial system 1800. The annotation system 1906operatively supplies a media overlay or supplementation (e.g., an imagefilter) to the messaging client application 1905 based on a geolocationof the client device 1802A or client device 1802B. In another example,the annotation system 1906 operatively supplies a media overlay to themessaging client application 1905 based on other information, such associal network information of the user of the client device 1802A orclient device 1802B. A media overlay may include audio and visualcontent and visual effects. Examples of audio and visual content includepictures, texts, logos, animations, and sound effects. An example of avisual effect includes color overlaying. The audio and visual content orthe visual effects can be applied to a media content item (e.g., aphoto) at the client device 1802A or client device 1802B. For example,the media overlay may include text that can be overlaid on top of aphotograph taken by the client device 1802A or client device 1802B. Inanother example, the media overlay includes an identification of alocation overlay (e.g., Venice beach), a name of an AR tutorial, a nameof show that provides AR tutorials, a name of a live event, or a name ofa merchant overlay (e.g., Beach Coffee House). In another example, theannotation system 1906 uses the geolocation of the client device 1802Aor client device 1802B to identify a media overlay that includes thename of a merchant at the geolocation of the client device 1802A orclient device 1802B. The media overlay may include other indiciaassociated with the merchant. The media overlays may be stored in thedatabase 1820 and accessed through the database server 1818.

In one example embodiment, the annotation system 1906 provides auser-based publication platform that enables users to select a body partand upload content associated with the body part for AR tutorials. Theuser may also specify circumstances under which a particular mediaoverlay should be offered to other users. The annotation system 1906generates a media overlay that includes the uploaded content andassociates the uploaded content with the selected body part to enable auser of the client device 1802A or client device 1802B to search for ARtutorial based on the body part.

In another example embodiment, the annotation system 1906 provides amerchant-based publication platform that enables merchants to select aparticular media overlay associated with a body part via a biddingprocess. For example, the annotation system 1906 associates the mediaoverlay of a highest bidding merchant with a corresponding body part fora predefined amount of time.

FIG. 20 is a schematic diagram illustrating data structures 2000 whichmay be stored in the database 1820 of the AR tutorial server system1808, according to certain example embodiments. While the content of thedatabase 1820 is shown to comprise a number of tables, it will beappreciated that the data could be stored in other types of datastructures (e.g., as an object-oriented database).

The database 1820 includes message data stored within a message table2014. The entity table 2002 stores entity data, including an entitygraph 2004. Entities for which records are maintained within the entitytable 2002 may include individuals, corporate entities, organizations,objects, places, events, etc. Regardless of type, any entity regardingwhich the AR tutorial server system 1808 stores data may be a recognizedentity. Each entity is provided with a unique identifier, as well as anentity type identifier (not shown). Database 1820 is termed a datastore, in accordance with some embodiments.

The entity graph 2004 furthermore stores information regardingrelationships and associations between entities. Such relationships maybe social, professional (e.g., work at a common corporation ororganization) interested-based or activity-based, merely for example.

The database 1820 also stores annotation data, in the example form offilters, in an annotation table 2012. Filters for which data is storedwithin the annotation table 2012 are associated with and applied to ARtutorials, videos (for which data is stored in a video table 2010),and/or images (for which data is stored in an image table 2008).Filters, in one example, are overlays that are displayed as overlaid onan AR tutorial, image, or video during presentation to a recipient user.Filters may be of varies types, including user-selected filters from agallery of filters presented to a sending user by the messaging clientapplication 1905 when the sending user is composing a message. Othertypes of filters include geolocation filters (also known as geo-filters)which may be presented to a sending user based on geographic location.For example, geolocation filters specific to a neighborhood or speciallocation may be presented within a user interface by the messagingclient application 1905, based on geolocation information determined bya GPS unit of the client device 1802A or client device 1802B. Anothertype of filer is a data filer, which may be selectively presented to asending user by the messaging client application 1905, based on otherinputs or information gathered by the client device 1802A or clientdevice 1802B during the message creation process. Example of datafilters include current temperature at a specific location, a currentpresenter of an AR tutorial, a current show providing AR tutorialcontent, a current body part, a current beauty product, a current speedat which a sending user is traveling, battery life for a client device1802A or client device 1802B, or the current time.

Other annotation data that may be stored within the image table 2008 isdata corresponding to Lenses® and/or media overlays. One example of amedia overlay is a real-time special effect and sound that may be addedto an image or video. The media overlay may include effects of an ARtutorial. In some embodiments, an AR tutorial 143 may be termed aLenses®. In some embodiments Lenses® comprise media overlays.

The video table 2010 stores video data which, in one embodiment, isassociated with messages, beauty data, or AR tutorials for which recordsare maintained within the message table 2014. Similarly, the image table2008 stores image data associated with messages for which message datais stored in the entity table 2002. The entity table 2002 may associatevarious annotations from the annotation table 2012 with various ARtutorials, beauty products, images, and videos stored in the image table2008 and the video table 2010.

The audio table 2016 stores audio data which, in one embodiment, isassociated with messages or AR tutorials for which records aremaintained within the message table 2014. Similarly, the audio table2016 stores audio data associated with messages or AR tutorials forwhich message data or AR tutorial data is stored in the entity table2002. The entity table 2002 may associate various annotations from theannotation table 2012 with various audio data stored in the audio table2016 or AR tutorials stored in the AR tutorial data 2020. Audio data maybe associated with video data, message data, story data, AR tutorialdata, beauty products, or other types of data via the entity graph 2004.In some embodiments, the audio table 2016 stores songs.

A collection table 2006 stores data regarding collections of ARtutorials, beauty products, messages and associated image, video, oraudio data, which are compiled into a collection (e.g., story,collection, make over, body part, presentation of images, or a gallery).The creation of a particular collection may be initiated by a particularuser (e.g., each user for which a record is maintained in the entitytable 2002). A user may create a user collection or “personal story” inthe form of a collection of content that has been created andsent/broadcast by that user. To this end, the user interface of themessaging client application 1905 may include an icon that isuser-selectable to enable a sending user to add specific content to hisor her user collection or personal story.

A collection may also constitute a “live story,” which is a collectionof content from multiple users that is created manually, automatically,or using a combination of manual and automatic techniques. For example,a contemporaneous collection or “live story” may constitute a curatedstream of user-submitted content from varies locations, events, beautyproducts, or body parts. Users whose client devices have locationservices enabled and are at a common location event at a particular timemay, for example, be presented with an option, via a user interface ofthe AR tutorial client application 1804A or AR tutorial clientapplication 1804B, to contribute content to a particular contemporaneouscollection or live story. The contemporaneous collection or live storymay be identified to the user by the AR tutorial client application1804A or AR tutorial client application 1804B, based on his or herlocation. The end result is a contemporaneous collection or “live story”told from a community perspective.

A further type of content collection is known as a location collectionor “location story”, which enables a user whose client device 1802A orclient device 1802B is located within a specific geographic location(e.g., on a college or university campus) to contribute to a particularcollection. In some embodiments, a contribution to a location collectionor location story may require a second degree of authentication toverify that the end user belongs to a specific organization or otherentity (e.g., is a student on the university campus). AR tutorial data2020 includes AR tutorial data 202 as disclosed herein. Beauty products2018 includes beauty product data 302 as disclosed herein. Beautyproducts 2018 may include stored image data of beauty products that maybe used to compare images of beauty products associated with an ARtutorial 143 and identify the beauty products associated with the ARtutorial 143 based on matching the images. In some embodiments, contentcollection may be known as a “make over” and be associated with a bodypart, a show that provides AR tutorial content, and/or a presenter.

FIG. 21 is a diagrammatic representation of the machine 2100 withinwhich instructions 2108 (e.g., software, a program, an application, anapplet, an app, or other executable code) for causing the machine 2100to perform any one or more of the methodologies discussed herein may beexecuted. For example, the instructions 2108 may cause the machine 2100to execute any one or more of the methods and/or functions describedherein. The instructions 2108 transform the general, non-programmedmachine 2100 into a particular machine 2100 programmed to carry out thedescribed and illustrated functions in the manner described. The machine2100 may operate as a standalone device or may be coupled (e.g.,networked) to other machines. In a networked deployment, the machine2100 may operate in the capacity of a server machine or a client machinein a server-client network environment, or as a peer machine in apeer-to-peer (or distributed) network environment. The machine 2100 maycomprise, but not be limited to, a server computer, a client computer, apersonal computer (PC), a tablet computer, a laptop computer, a netbook,a set-top box (STB), a PDA, an entertainment media system, a cellulartelephone, a smart phone, a mobile device, a wearable device (e.g., asmart watch), a smart home device (e.g., a smart appliance), other smartdevices, a web appliance, a network router, a network switch, a networkbridge, or any machine capable of executing the instructions 2108,sequentially or otherwise, that specify actions to be taken by themachine 2100. Further, while only a single machine 2100 is illustrated,the term “machine” shall also be taken to include a collection ofmachines that individually or jointly execute the instructions 2108 toperform any one or more of the methodologies discussed herein. In someembodiments, an apparatus of the machine 2100 is configured to performone or more of the functions as disclosed herein.

The machine 2100 may include processors 2102, memory 2104, and I/Ocomponents 2138, which may be configured to communicate with each othervia a bus 2140. In an example embodiment, the processors 2102 (e.g., aCentral Processing Unit (CPU), a Reduced Instruction Set Computing(RISC) Processor, a Complex Instruction Set Computing (CISC) Processor,a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), anASIC, a Radio-Frequency Integrated Circuit (RFIC), another processor, orany suitable combination thereof) may include, for example, a processor2106 and a processor 2110 that execute the instructions 2108. The term“processor” is intended to include multi-core processors that maycomprise two or more independent processors (sometimes referred to as“cores”) that may execute instructions contemporaneously. Although FIG.21 shows multiple processors 2102, the machine 2100 may include a singleProcessor with a single core, a single Processor with multiple cores(e.g., a multi-core Processor), multiple processors with a single core,multiple processors with multiples cores, or any combination thereof.

The memory 2104 includes a main memory 2112, a static memory 2114, and astorage unit 2116, both accessible to the processors 2102 via the bus2140. The memory 2104, the static memory 2114, and storage unit 2116store the instructions 2108 embodying any one or more of themethodologies or functions described herein. The instructions 2108 mayalso reside, completely or partially, within the main memory 2112,within the static memory 2114, within machine-readable medium 2118within the storage unit 2116, within at least one of the processors 2102(e.g., within the Processor's cache memory), or any suitable combinationthereof, during execution thereof by the machine 2100.

The I/O components 2138 may include a wide variety of components toreceive input, provide output, produce output, transmit information,exchange information, capture measurements, and so on. The specific I/Ocomponents 2138 that are included in a particular machine will depend onthe type of machine. For example, portable machines such as mobilephones may include a touch input device or other such input mechanisms,while a headless server machine will likely not include such a touchinput device. It will be appreciated that the I/O components 2138 mayinclude many other components that are not shown in FIG. 21 . In variousexample embodiments, the I/O components 2138 may include user outputcomponents 2124 and user input components 2126. The user outputcomponents 2124 may include visual components (e.g., a display such as aplasma display panel (PDP), a light emitting diode (LED) display, aliquid crystal display (LCD), a touch screen, a projector, or a cathoderay tube (CRT)), acoustic components (e.g., speakers), haptic components(e.g., a vibratory motor, resistance mechanisms), other signalgenerators, and so forth. The user input components 2126 may includealphanumeric input components (e.g., a keyboard, a touch screenconfigured to receive alphanumeric input, a photo-optical keyboard, orother alphanumeric input components), point-based input components(e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, oranother pointing instrument), tactile input components (e.g., a physicalbutton, a touch screen that provides location and/or force of touches ortouch gestures, or other tactile input components), audio inputcomponents (e.g., a microphone), and the like.

In further example embodiments, the I/O components 2138 may includebiometric components 2128, motion components 2130, environmentalcomponents 2132, or position components 2134, among a wide array ofother components. For example, the biometric components 2128 includecomponents to detect expressions (e.g., hand expressions, facialexpressions, vocal expressions, body gestures, or eye-tracking), measurebiosignals (e.g., blood pressure, heart rate, body temperature,perspiration, or brain waves), identify a person (e.g., voiceidentification, retinal identification, facial identification,fingerprint identification, or electroencephalogram-basedidentification), and the like. The motion components 2130 includeacceleration sensor components (e.g., accelerometer), gravitation sensorcomponents, rotation sensor components (e.g., gyroscope). Theenvironmental components 2132 include, for example, one or cameras (withstill image/photograph and video capabilities), illumination sensorcomponents (e.g., photometer), temperature sensor components (e.g., oneor more thermometers that detect ambient temperature), humidity sensorcomponents, pressure sensor components (e.g., barometer), acousticsensor components (e.g., one or more microphones that detect backgroundnoise), proximity sensor components (e.g., infrared sensors that detectnearby objects), gas sensors (e.g., gas detection sensors to detectionconcentrations of hazardous gases for safety or to measure pollutants inthe atmosphere), or other components that may provide indications,measurements, or signals corresponding to a surrounding physicalenvironment. The position components 2134 include location sensorcomponents (e.g., a GPS receiver Component), altitude sensor components(e.g., altimeters or barometers that detect air pressure from whichaltitude may be derived), orientation sensor components (e.g.,magnetometers), and the like.

Communication may be implemented using a wide variety of technologies.The I/O components 2138 further include communication components 2136operable to couple the machine 2100 to a network 2120 or devices 2122via respective coupling or connections. For example, the communicationcomponents 2136 may include a network interface Component or anothersuitable device to interface with the network 2120. In further examples,the communication components 2136 may include wired communicationcomponents, wireless communication components, cellular communicationcomponents, Near Field Communication (NFC) components, Bluetooth®components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and othercommunication components to provide communication via other modalities.The devices 2122 may be another machine or any of a wide variety ofperipheral devices (e.g., a peripheral device coupled via a USB).

Moreover, the communication components 2136 may detect identifiers orinclude components operable to detect identifiers. For example, thecommunication components 2136 may include Radio Frequency Identification(RFID) tag reader components, NFC smart tag detection components,optical reader components (e.g., an optical sensor to detectone-dimensional bar codes such as Universal Product Code (UPC) bar code,multi-dimensional bar codes such as Quick Response (QR) code, Azteccode, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2Dbar code, and other optical codes), or acoustic detection components(e.g., microphones to identify tagged audio signals). In addition, avariety of information may be derived via the communication components2136, such as location via Internet Protocol (IP) geolocation, locationvia Wi-Fi® signal triangulation, location via detecting an NFC beaconsignal that may indicate a particular location, and so forth.

The various memories (e.g., main memory 2112, static memory 2114, and/ormemory of the processors 2102) and/or storage unit 2116 may store one ormore sets of instructions and data structures (e.g., software) embodyingor used by any one or more of the methodologies or functions describedherein. These instructions (e.g., the instructions 2108), when executedby processors 2102, cause various operations to implement the disclosedembodiments.

The instructions 2108 may be transmitted or received over the network2120, using a transmission medium, via a network interface device (e.g.,a network interface Component included in the communication components2136) and using any one of several well-known transfer protocols (e.g.,hypertext transfer protocol (HTTP)). Similarly, the instructions 2108may be transmitted or received using a transmission medium via acoupling (e.g., a peer-to-peer coupling) to the devices 2122.

FIG. 22 is a block diagram illustrating a software architecture 2204,which can be installed on any one or more of the devices describedherein. The software architecture 2204 is supported by hardware such asa machine 2202 that includes processors 2220, memory 2226, and I/Ocomponents 2238. Machine 2202 is the same or similar as machine 2100, inaccordance with some embodiments. In this example, the softwarearchitecture 2204 can be conceptualized as a stack of layers, where eachlayer provides a particular functionality. The software architecture2204 includes layers such as an operating system 2212, libraries 2210,frameworks 2208, and applications 2206. Operationally, the applications2206 invoke API calls 2250 through the software stack and receivemessages 2252 in response to the API calls 2250.

The operating system 2212 manages hardware resources and provides commonservices. The operating system 2212 includes, for example, a kernel2214, services 2216, and drivers 2222. The kernel 2214 acts as anabstraction layer between the hardware and the other software layers.For example, the kernel 2214 provides memory management, Processormanagement (e.g., scheduling), Component management, networking, andsecurity settings, among other functionality. The services 2216 canprovide other common services for the other software layers. The drivers2222 are responsible for controlling or interfacing with the underlyinghardware. For instance, the drivers 2222 can include display drivers,camera drivers, BLUETOOTH® or BLUETOOTH® Low Energy drivers, flashmemory drivers, serial communication drivers (e.g., Universal Serial Bus(USB) drivers), WI-FI® drivers, audio drivers, power management drivers,and so forth.

The libraries 2210 provide a low-level common infrastructure used by theapplications 2206. The libraries 2210 can include system libraries 2218(e.g., C standard library) that provide functions such as memoryallocation functions, string manipulation functions, mathematicfunctions, and the like. In addition, the libraries 2210 can include APIcalls 2250 libraries such as media libraries (e.g., libraries to supportpresentation and manipulation of various media formats such as MovingPicture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC),Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC),Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group(JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries(e.g., an OpenGL framework used to render in two dimensions (2D) andthree dimensions (3D) in a graphic content on a display), databaselibraries (e.g., SQLite to provide various relational databasefunctions), web libraries (e.g., WebKit to provide web browsingfunctionality), and the like. The libraries 2210 can also include a widevariety of other 2228 libraries to provide many other APIs to theapplications 2206.

The frameworks 2208 provide a high-level common infrastructure that isused by the applications 2206. For example, the frameworks 2208 providevarious graphical user interface (GUI) functions, high-level resourcemanagement, and high-level location services. The frameworks 2208 canprovide a broad spectrum of other APIs that can be used by theapplications 2206, some of which may be specific to a particularoperating system or platform.

In an example embodiment, the applications 2206 may include a home 2236application, a contacts 2230 application, a browser 2232 application, abook reader 2234 application, a location application 2242, a media 2224application, a messaging 2227 application, a game 2228 application, anda broad assortment of other applications such as a third-partyapplication 2240. The applications 2206 are programs that executefunctions defined in the programs. Various programming languages can beemployed to create one or more of the applications 2206, structured in avariety of manners, such as object-oriented programming languages (e.g.,Objective-C, Java, or C++) or procedural programming languages (e.g., Cor assembly language). In a specific example, the third-partyapplication 2240 (e.g., an application developed using the ANDROID™ orIOS™ software development kit (SDK) by an entity other than the vendorof the particular platform) may be mobile software running on a mobileoperating system such as IOS™, ANDROID™ WINDOWS® Phone, or anothermobile operating system. In this example, the third-party application2240 can invoke the API calls 2250 provided by the operating system 2212to facilitate functionality described herein.

FIG. 23 illustrates components of editing an AR tutorial, in accordancewith some embodiments. Camera 130, AR tutorial state 148, recording 150,playing back 152, AR tutorial 143, paused 154, mobile device 146 are thesame or similar as the elements with like numbers of FIG. 1 . Zoom level208, presenter 236A, 236B, 236C, visible area 206, playback speed value210, start time 212, end time 214, are the same or similar as theelements with like numbers of FIG. 2 . Playback speed bar 414 is thesame or similar as playback speed bar 414. Screen 2300A, 2300B, and2300C are the same or similar as screen 132. Finger 810 is the same orsimilar as finger 810 of FIG. 8 . Thumb 2306 is a digit of a humanbeing, e.g., the presenter 236B.

As illustrated in screen 2300A, camera 130 is capturing live images 134of FIG. 1 of the presenter 236A as the presenter 236A generates ARtutorial 143 or the presenter 236A is playing back 152 AR tutorial 143.As illustrated in screen 2300A, AR tutorial 143 is paused 154 andpresenter 236A selecting play icon 2302 will change the AR tutorialstate 148 to playing back 152 or recording 150.

The presenter 236A of screen 2300A selects to edit 2304. Edit 2304 is acommand 112 of FIG. 1 . Edit 2304 may be selected by moving the thumb2306 and finger 810B on the screen 2300B, moving the finger 810C on thescreen 2300C, or moving finger 810D on screen 2300D. Edit 2304 may beinvoked in a different way.

Screen 2300B illustrates edit zoom 2350. The presenter 236B is editingzoom level 208, which indicates a value of “150%”, by moving her thumb2306 and finger 810B. Screen 2300C illustrates edit playback speed 2352.Playback speed value 210 indicates a playback speed value 210 for ARtutorial 143. As illustrated, playback speed value 210 indicates “1.3×”or 1.3 times a normal or standard playback speed. The presenter 236Cmoves her finger 810C to move playback speed indicator 2308 alongplayback speed bar 414 to edit the playback speed value 210. Playbackspeed bar 414 indicates a range of playback speed values 210 that may beselected by moving playback speed indicator 2308.

Screen 2300D illustrates edit video area 2354. The presenter 236D isediting visible 206 by moving her finger 810D. Whole video 2310indicates the size of the whole video, which may be playing back 152 orrecording 150. Visible area 206 indicates the portion of the whole video2310 that is being displayed on the screen 2300D. In some embodiments along hold of finger 810D invokes edit video area 2354.

Each of zoom level 208, playback speed value 210, and visible area 206is associated with a start time 212 and end time 214. Zoom level 208,playback speed value 210, and visible area 206 are edited at a starttime 212 and the zoom level 208, playback speed value 210, and visible206 stay at the zoom level 208, playback speed value 210, and visiblearea 206 until an end time 214. The end time 214 is determined when thezoom level 208, playback speed value 210, and visible area 206 arechanged or when the AR tutorial 143 enter a new step N 204. In someembodiments, the end time 214 is determined based on AR tutorial module102 of FIG. 1 , which resets the zoom level 208, playback speed value210, and visible area 206 at the beginning of a step N 204. In someembodiments, start time 212 and/or end time 214 may be set or resetdifferently.

Edit zoom 2350, edit playback speed 2352, edit video area 2354 arecommands 112 as disclosed in conjunction with FIG. 1 . The presentationof the icons may be different in position and appearance. In someembodiments, zoom level 208, playback speed value 210, and/or visiblearea 206 are only displayed on the screen 2300B, screen 2300C, screen2300D, respectively, when the values are being edited. In someembodiments, edit AR tutorial module 158 is configured to respond tocommands 112 edit zoom 2350, edit playback speed 2352, and/or edit videoarea 2354 as disclosed in conjunction with FIG. 23 .

FIG. 24 illustrates components of adding a step 2400 in an AR tutorial143, in accordance with some embodiments. Camera 130, AR tutorial state148, recording 150, playing back 152, paused 154, mobile device 146, andAR tutorial 143 are the same or similar as elements with like numbers inFIG. 1 . Start time 212, end time 214, and presenter 236 are the same orsimilar as elements with like numbers in FIG. 2 . Finger 810 is the sameas or similar to finger 810 of FIG. 8 . Screen 2400A and screen 2400Bare the same as or similar to screen 132. Step mark 426A, 426B, play bar422, current play 424A, 424B, and play bar 422 are the same or similaras elements with like numbers of FIG. 4 . Step 1 1608A, step 2 1608B,and step 3 1608C are the same or similar as step 1 1608A, step 2 1608B,and step 3 1608C of FIG. 16 .

Edit menu 2402 may be present during the recording 150 or playing back152 of AR tutorial 143. In some embodiments, edit menu 2402 is displayedbased on a command 112 of FIG. 1 of the presenter 236. For example, thepresenter 236 may perform a command 112 that invokes the edit menu 2402such as a left or right swipe of finger 810, a double-tap of finger 810,and so forth. Invoking the edit menu 2402 is a command 112 of FIG. 1 .Edit menu 2402 may include one or more additional commands 112, such asadd pause 2504 of FIG. 25 . Edit menu 2402 may be displayed differentlyor not at all.

Play bar 422 indicates a length of the AR tutorial 143 or a lengthavailable to record the AR tutorial 143. Step marks 426A, 426B indicatewhere steps such as step N 204 begin and end. Current play 424A, 424Bindicate a current play location within the play bar 422 of the ARtutorial 143, which may be playing back 152, recording 150, or paused154.

As illustrated, presenter 236 selects add step icon 2404 with her finger810 to invoke command add step 2406. In response, respond to commandsmodule 110 of FIG. 1 displays screen 2400B. A new step mark 426A isdisplayed to indicate the creation of a new step such as step 1 1608A,step 2 1608B, and step 3 1608C. Step 1 1608A, step 2 1608B, and step 31608C begin and end at step marks 426. The presenter 236 invoking addstep 2406 adds step mark 426B where the current play 424A is located.Invoking add step 2406 ends step 2 1608B and begins step 3 1608C.

In some embodiments, AR tutorial 143 will move to a paused 154 ARtutorial state 148 after the invocation of add step 2406. Thepresentation of the screen 2400A and screen 2400B may be different. Forexample, edit menu 2402 may be displayed in a different location or maybe accessed by a left swipe, right swipe, up swipe, and/or down swipe.Command add step 2406 may be entered differently by the presenter 236 asdisclosed in conjunction with FIG. 1 . For example, with a mouse orvoice command. Edit menu 2402 and add step icon 2404 are UI items 234 ofFIG. 2 .

FIG. 25 illustrates components of creating a pause in an AR tutorial, inaccordance with some embodiments. Camera 130, AR tutorial state 148,recording 150, playing back 152, paused 154, mobile device 146, and ARtutorial 143 are the same or similar as elements with like numbers inFIG. 1 . Start time 212, end time 214, and presenter 236 are the same orsimilar as elements with like numbers in FIG. 2 . Finger 810 is the sameor similar as finger 810 of FIG. 8 . Screen 2500A and screen 2500B arethe same or similar as screen 132. Step mark 426, play bar 422, currentplay 424A, 424B, and play bar 422 are the same or similar as elementswith like numbers of FIG. 4 . Step 1 1608A and step 2 1608B are the sameor similar as step 1 1608A and step 2 1608B of FIG. 16 . Edit menu 2402is the same or similar as edit menu 2402.

The add pause icon 2502 and the pause mark 2506 may be displayeddifferently. Command add pause 2504 may be invoked differently by thepresenter 236 such as is disclosed in conjunction with FIG. 1 andherein. The add pause 2504 command is a command 112 as disclosed inconjunction with FIG. 1 . Pause mark 2506 is a pause 220 as disclosed inconjunction with FIG. 2 .

As illustrated, presenter 236 of screen 2500A selects add pause icon2502 to invoke command add pause 2504. A new pause mark 2506 isdisplayed to indicate the creation of a new pause such as pause 220 ofstep N 204 of FIG. 2 . Pause mark 2506 indicates a start time 212 whenthe playing of AR tutorial state 148 will enter a paused 154 AR tutorialstate 148. In some embodiments, a play icon 2302, such as is disclosedin FIG. 23 , will be displayed to indicate that a selection of the playicon 2302 will return AR tutorial state 148 to recording 150 or playingback 152. In some embodiments, text and/or hotlinks may be associatedwith a pause 220. For example, the AR tutorial 143 may enter paused 154AR tutorial state 148 and display text such as “View Product Page?” witha hotlink to the beauty product 224 page.

FIGS. 26 and 27 are disclosed in conjunction with one another. FIG. 26illustrates components of adding a body part 2702, in accordance withsome embodiments. FIG. 27 illustrates components of adding a body part2702, in accordance with some embodiments. Camera 130, mobile device146, and AR tutorial 143 are the same or similar as elements with likenumbers in FIG. 1 . Presenter 236 is the same or similar as presenter236 of FIG. 2 . Finger 810 is the same or similar as finger 810 of FIG.8 . Screen 2600A, screen 2600B, screen 2600C, screen 2600D, screen2600E, and screen 2700 are the same or similar as screen 132. Step mark426, play bar 422, current play 424, play bar 422, and back 428 are thesame or similar as elements with like numbers of FIG. 4 . Step 2 1608Bis the same or similar as step 2 1608B of FIG. 16 . Edit menu 2402 isthe same or similar as edit menu 2402.

As illustrated, presenter 236 selects with her finger 810A add body particon 2602 to invoke add body part 2604. Add body part 2604 is a command112 of FIG. 1 . Add body part 2604 enables a presenter 236 to add a bodypart 222 for a step N 204 of AR tutorial 143. Respond to commands module110 and/or record AR tutorial module 158 responds to the selection ofinvocation of add body part 2604 by presenting screen 2600B or screen2700. Screen 2600B enables the presenter 236 to select a general area ofthe body 2606. Body 2606 is an image of a body to facilitate thepresenter 236 selecting a body part 222. In some embodiments, add bodypart 2604 defaults to adding a body part 222 to a current step of the ARtutorial 143. For example, as illustrated the current play 424 bar iswithin step 2 1608B, so the command add body part 2604 adds the bodypart 222 for step 2 1608B of AR tutorial 143.

The presenter 236 can invoke command body part select 2608A by tapping aportion of body 2606. Respond to commands module 110 and/or record ARtutorial module 158 respond to body part select 2608A by displaying thepresenter 236 with the portion of the body 2606 selected by thepresenter 236 enlarged to occupy the screen 2600C.

The presenter 236 may select a portion of the presenter 236 on screen2600C. As illustrated, presenter 236 selects a right eye to invokeselect body part 2608B command. Select body part 2608A, 2608B, andconfirm 2612 are commands 112 as disclosed in conjunction with FIG. 1 .In some embodiments, presenter 236 may invoke add body part 2604 andthen select a body part from presenter 236 that is already beingdisplayed on the screen 2600A. Respond to commands module 110 and/orrecord AR tutorial module 158 respond to the invocation of select bodypart 2608B by displaying the selected body part 222 to occupy the screen2600D. The presenter 236 may invoke confirm 2612 to confirm the bodypart 222 selection. For example, the presenter 236 may select OK 2610.Respond to commands module 110 and/or record AR tutorial module 158respond to the invocation of confirm 2612 by setting body part 222 ofstep 2 1608B as a right eye. In some embodiments, add body part icon2602 is changed to indicate the body part 222 in screen 2600E, which mayindicate the body part 222 selected. Step 1 1608A, step 2 1608B, andstep 3 1608C may display different body parts 222. The presenter 236 mayselect back 428 to return to an earlier screen.

Returning to screen 2700, in some embodiments, respond to commandsmodule 110 and/or record AR tutorial module 158 respond to theinvocation of add body part 2604, by displaying screen 2700. Thepresenter 236 invoke select body part 2608C with her finger 810 toselect a body part 222 of the body parts 2702 displayed. Respond tocommands module 110 and/or record AR tutorial module 158 respond to theinvocation of select body part 2608C with displaying screen 2300D or2600E.

FIG. 28 illustrates components of adding a body part, in accordance withsome embodiments. Camera 130, mobile device 146, and AR tutorial 143 arethe same or similar as elements with like numbers in FIG. 1 . Presenter236 is the same or similar as presenter 236 of FIG. 2 . Finger 810 isthe same or similar as finger 810 of FIG. 8 . Screen 2800A and screen2800B are the same or similar as screen 132. Step mark 426, play bar422, current play 424, play bar 422, and back 428 are the same orsimilar as elements with like numbers of FIG. 4 . Step 2 1608B is thesame or similar as step 2 1608B of FIG. 16 . Edit menu 2402 is the sameor similar as edit menu 2402. Add body part 2604 is the same or similaras add body part 2604 of FIG. 26 . Add body part icon 2602 is the sameor similar as add body part icon 2602.

As illustrated, the presenter 236 places her finger to her right eye toinvoke add body part 2604. Respond to commands module 110 and/or recordAR tutorial module 158 respond to the invocation of add body part 2604by making the right eye the body part 222 of step 2 1608B. Add body particon 2602 is changed to an eye to indicate the selection of the eye. Insome embodiments, presenter 236 invokes add body part 2604 by firstselecting add body part icon 2602 and then selecting the body part 222with a gesture such as putting a finger up to the right eye. In someembodiments, the presenter 236 invoking command back 428 in screen 2800Bwill return to screen 2800A. The presentation of the icons and thepresenter 236 may be different than illustrated in screen 2800A andscreen 2800B.

In some embodiments, AR tutorial module 102 will automatically determinewhich body part 222 of the presenter 236 the presenter 236 is working onfor the AR tutorial 143. For example, when step 2 1608B is begun ARtutorial module 102 may determine which body part 222 is having a beautyproduct 224 applied to it or which body part 222 has activity on or nearit. In some embodiments, AR tutorial module 102 may confirm with thepresenter 236 the body part 222 that was automatically detected. In someembodiments, AR tutorial module 102 determines the body part 222 in adifferent way such as based on the beauty product 224 of step 2 1608B.

FIG. 29 illustrates components of adding a beauty product, in accordancewith some embodiments. Camera 130, mobile device 146, and AR tutorial143 are the same or similar as elements with like numbers in FIG. 1 .Presenter 236 is the same or similar as presenter 236 of FIG. 2 . Finger810 is the same or similar as finger 810 of FIG. 8 . Screen 2900A,screen 2900B, screen 2900C, screen 2900D, and screen 2900E are the sameor similar as screen 132. Step mark 426, play bar 422, current play 424,play bar 422, and back 428 are the same or similar as elements with likenumbers of FIG. 4 . Step 2 1608B is the same or similar as step 2 1608Bof FIG. 16 . Edit menu 2402 is the same or similar as edit menu 2402 ofFIG. 24 .

As illustrated, presenter 236 of screen 2900A invokes command 112 addbeauty product 2904 by selecting add beauty product icon 2902 with herfinger 810A. In response, AR tutorial module 102 displays screen 2900Bwith a beauty product list 2908A that the presenter 236 may select orscroll through with her finger 810B. The beauty product list 2908Acomprises images of beauty products 224 of FIG. 2 where the images maybe images of beauty product 326 of FIG. 3 . In some embodiments, thebeauty product list 2908A that are displayed are dependent on the bodypart 222. In some embodiments, presenter 236 selects beauty products 224for a different body part 222 by scrolling to the left or right, such asselecting right arrow 2906, which invokes a command 112 scroll.

Back 428 offers the presenter 236 the option to return to a previousscreen. As illustrated, presenter 236 invokes beauty product select2910A by tapping with her finger 810B an image of a beauty product inthe beauty product list 2908A. In response, AR tutorial module 102displays screen 2900C. Screen 2900C displays beauty product list 2908Bfor different types of beauty products 224 that are in a same categoryas the beauty product 224 selected from screen 2900B. Presenter 236 mayselect different manufacturers 2911, which will provide a beauty productlist 2908B from different manufactures. As illustrated, presenter 236invokes command 112 beauty product select 2910B by selecting an image ofa beauty product 224 with her finger 810C that is a variety of brush. Inresponse, AR tutorial module 102 displays screen 2900D.

Screen 2900D provides the option for the presenter 236 to invoke command112 confirm 2918 by selecting OK 2914, to select a hotlink 2916 to see aweb site regarding the beauty product 224 where the website may includesuch information as product information, purchasing information, and soforth. As illustrated, presenter 236 invoke command 112 confirm 2918and, in response, AR tutorial module 102 displays screen 2900E.

As illustrated, AR tutorial module 102 adjusts zoom level 208 andvisible area 206 and changes add beauty product icon 2902 to beautyproduct icon 2920. AR tutorial module 102 may automatically select aneye as the body part 222 as the beauty product 224 is meant for an eye,which may be determined based on beauty product data 302 that mayinclude body part 308. AR tutorial module 102 uses the body part 308 todetermine the body part 222 for step 2 1608A as an eye and to adjust thezoom level 208 and visible area 206 to the eye of the presenter 236, inaccordance with some embodiments. In some embodiments if the body part222 is already determined before a beauty product 224 is selected, thenAR tutorial module 102 displays beauty product list 2908A for theselected body part 222. In some embodiments, invoking commands 112 maybe performed in different ways as disclosed in conjunction with FIG. 1and herein Screen 2900A, screen 2900B, screen 2900C, screen 2900D, andscreen 2900E may be presented differently with additional information orsome information not presented. Add beauty product icon 2902, back 428,manufacturer 2911, edit menu 2402, OK 2914, edit 2912, and hotlink 2916may be displayed differently or not at all.

FIG. 30 illustrates components of adding a beauty product, in accordancewith some embodiments. Camera 130, mobile device 146, and AR tutorial143 are the same or similar as elements with like numbers in FIG. 1 .Presenter 236 is the same or similar as presenter 236 of FIG. 2 . Finger810 is the same or similar as finger 810 of FIG. 8 . Screen 3000A andscreen 3000B are the same or similar as screen 132. Step mark 426, playbar 422, current play 424, play bar 422, and back 428 are the same orsimilar as elements with like numbers of FIG. 4 . Step 2 1608B is thesame or similar as step 2 1608B of FIG. 16 . Edit menu 2402 is the sameor similar as edit menu 2402 of FIG. 24 .

As illustrated in screen 3000A, presenter 236 holds a beauty product 224near her eye to invoke command 112 add beauty product 2904, and, inresponse, AR tutorial module 102 identifies the beauty product 224 anddisplays screen 3000B. AR tutorial module 102 may have identified beautyproduct 224 from the images of beauty product 224 as disclosed inconjunction with FIG. 36 . In some embodiments, presenter 236 selectsadd beauty product icon 3002 to invoke command 112 add beauty product2904. Screen 3000B illustrates that beauty product 224 was identifiedand an icon of the beauty product 224 was retrieved to display in theedit menu 2402 as beauty product icon 3004. In some embodiments, editmenu 2402 is not changed or changed in a different way such as boldingto indicate that a beauty product 224 has been selected for step 2 1608Bof AR tutorial 143. Referring to FIG. 16 , beauty product 224 isindicated as the beauty product 1606B for step 2 1608B of the ARtutorial 143. The presenter 236 may select back 428 to return to aprevious screen, in accordance with some embodiments. Back 428 may beselected to return to a previous screen.

FIG. 31 illustrates components of adding or editing motion, inaccordance with some embodiments. Camera 130, mobile device 146, and ARtutorial 143 are the same or similar as elements with like numbers inFIG. 1 . Presenter 236 is the same or similar as presenter 236 of FIG. 2. Finger 810 is the same or similar as finger 810 of FIG. 8 . Screen3100A, screen 3100B, screen 3100C, and screen 3100D are the same orsimilar as screen 132. Step mark 426, play bar 422, current play 424,play bar 422, and back 428 are the same or similar as elements with likenumbers of FIG. 4 . Step 2 1608B is the same or similar as step 2 1608Bof FIG. 16 . Edit menu 2402 is the same or similar as edit menu 2402 ofFIG. 24 .

As illustrated, the beauty product 224 and body part 222 have alreadybeen determined for step 2 1608B as illustrated in the edit menu 2402.Determine motion module 122 as disclosed in conjunction with FIG. 1 andherein determines tutorial motion 216 based on movement of beautyproduct 224 by the presenter 236, in accordance with some embodiments.As illustrated in screen 3100A, presenter 236 selects edit motion icon3102 with her finger 810A to invoke command 112 edit motion 3104A and,in response, AR tutorial module 102 such as determine motion module 122displays screen 3100B or screen 3100C. Illustrated in screen 3100B istutorial motion 216, edit 3106, and back 428. Presenter 236 may selectback 428 to return to a previous screen 132, e.g., screen 3100A. Asillustrated, presenter 236 selects edit 3106 to invoke command 112 editmotion 3104B, and, in response, AR tutorial module 102 displays screen3100C. In some embodiments, AR tutorial module 102 displays screen 3100Cdirectly in response to command 112 edit motion 3104A.

As illustrated, presenter 236 swipes her finger 810B to create new ARmotion 3108. In some embodiments, the presenter 236 deletes Tutorialmotion 216 or portions of Tutorial motion 216 by selecting tutorialmotion 216 and then a button appears to delete. In some embodiments,presenter 236 can delete tutorial motion 216 or portions of tutorialmotion 216 in a different way. In some embodiments, presenter 236 canmove tutorial motion 216 such as by a long press on tutorial motion 216and dragging.

As illustrated, presenter 236 selects done 3110, which invokes doneediting 3112, after adding new AR motion 3108 to tutorial motion 216and, in response, AR tutorial module 102 displays screen 3100D. Screen3100D illustrates tutorial motion 216, which include tutorial motion 216of screen 3100C with new AR motion 3108. Edit motion icon 3102 may bechanged to present an indication of the tutorial motion 216. In someembodiments, presenter 236 can select whether to display tutorial motion216. In some embodiments, tutorial motion 216 is displayed differentlyor not at all in the edit menu 2402.

FIG. 32 illustrates a system 3200 for creating an AR tutorial, inaccordance with some embodiments. Illustrated in FIG. 32 is menus andicons 3202, work area 3204, preview 3206, camera 130, screen 132, andcomputing device 3208. The camera 130 and screen 132 may be the same orsimilar as like numbered elements of FIG. 1 . Computing device 3208 maybe the same or similar as mobile device 146 where the computing device3208 includes computing devices that may be considered to not be mobile.Menus and icons 3202 present options for the presenter (not illustrated)to edit AR tutorial 143 of FIG. 1 . For example, the items of UI items234 may be presented in menus and icons 3202. Work area 3204 is an areafor the presenter to work on AR tutorial 143. For example, work area3204 displays a larger view of tutorial motion 216 of FIG. 31 to enablethe presenter to edit the tutorial motion 216. Preview 3206 presents ofa preview of the AR tutorial 143 such as screen 132, screen 2300A,screen 2300B, screen 2300C, and screen 2300D of FIG. 23 , screen 2400Aand screen 2400B of FIG. 24 , screen 2500A and screen 2500B of FIG. 25 ,screen 2600A, screen 2600B, screen 2600C, screen 2600D, and screen 2600Eof FIG. 26 , screen 2700 of FIG. 27 , screen 2800A and screen 2800B ofFIG. 28 , Screen 2900A, screen 2900B, screen 2900C, screen 2900D, andscreen 2900E of FIG. 29 , screen 3000A and screen 3000B of FIG. 30 , andscreen 3100A, screen 3100B, and screen 3100C of FIG. 31 . Computingdevice 3208 may be a machine 2100, in accordance with some embodiments.In some embodiments, system 3200 and/or mobile device 146 is a virtualreality headset.

FIG. 33 illustrates a method 3300 of adding beauty products to AR beautyproduct tutorials, in accordance with some embodiments. The method 3300begins at operation 3302 with accessing video data comprising images ofa presenter creating a tutorial where the tutorial depicts the presenterapplying a beauty product to a body part of the presenter. Referring toFIG. 1 , AR tutorial module 102 may access AR tutorial data 202, whichincludes AR tutorial video. Additionally, as disclosed in conjunctionwith FIGS. 1-3, 23-45 presenter 236 is recording AR tutorial 143.

The method 3300 continues at operation 3304 with processing the videodata to identify changes to the body part of the presenter from anapplication of the beauty product. For example, as disclosed inconjunction with FIG. 1 , determine effects module 128 is configured todetermine the AR effects 219 of FIGS. 2, 4, 5A, 5B, 5C, 8, 11, 12, 13,and 14 from tutorial effects 218 and/or AR tutorial video 228. FIGS. 5A,5B, and 5C illustrate an example of adding AR effects 219A, 219B, 219C,to a body part 222 that is a right eye of a user 238 from tutorialeffects 218A, which are the changes to a right eye of a presenter 236from applying beauty product 224B with beauty product 224A. FIGS. 5A,5B, and 5C illustrate the progression of the application of the eyeshadow where the changes to the right eye of the user 238 with the AReffects 219A, 219B, and 219C are synchronized with the changes to theright eye of the presenter 236, which are indicated by tutorial effects218A, 218B, and 218C.

The method 3300 continues at operation 3306 with in response toidentifying changes to the body part of the presenter from theapplication of the beauty product, processing the video data to identifythe beauty product. In one example as disclosed in conjunction with FIG.34 match image module 3402, match name module 3404, identify RFID module3406, and/or identify product code module 3408 may use tutorial effects218 of FIG. 2 to identify the beauty product 224. In an example,presenter 236 of FIG. 36 selects add beauty product icon 3602 to invokeadd beauty product 3604 where the beauty product 224 is identified asdisclosed in conjunction with screen 3600B. In another example, thepresenter 236 selects with their finger 810 add beauty product icon4106, which invokes scan beauty product 4108 command that results thebeauty product 224 being identified.

The method 3300 continues at operation 3308 with retrieving informationregarding the beauty product. In one example, identify produce codemodule 3408 identifies a product code 3610 from screen 3600B or screen4200A and matches the product code 3610 to a stored product code asdisclosed in conjunction with FIG. 34 .

The method 3300 continues at operation 3310 causing presentation ofinformation regarding the beauty product on a display device. In oneexample, screen 3700 of FIG. 37 is displayed. In another example, screen3800 of FIG. 38 is displayed. In another example, screen 4000 of FIG. 40is displayed. In another example, screen 4200B of FIG. 42 is displayed.In another example, in screen 2900E of FIG. 29 , beauty product icon2920 is displayed after the beauty product 224 is identified.

The method 3300 continues at operation 3312 with storing, within a datastore, first association data associating the beauty product with thetutorial. For example, AR tutorial module 102 may generate a record forthe beauty product 224 of FIG. 2 as beauty product data 302 withassociation data 332.

The method 3300 optionally includes identifying the beauty product basedon the changes to the body part of the presenter. For example, asdisclosed in conjunction with FIG. 34 match image module 3402, matchname module 3404, and identify product code module 3408 may basematching on a body part associated with a step n 204 of the AR tutorial143.

The method 3300 optionally includes where the changes to the body partof the presenter comprise a color change of the body part or a change oftexture of the body part. For example, as disclosed in conjunction withFIGS. 1-3 , tutorial effects 219 may be determined by determine effectsmodule 128 and tutorial effect 219 may include a color and texture.

The method 3300 optionally includes where the information regarding thebeauty product comprises one or more of: a name of the beauty product,the body part associated with the beauty product, a manufacturer of thebeauty product, a website associated with the beauty product, an imageof the beauty product, a product code of the beauty product, and anindication of ratings of the beauty product. For example, as disclosedin conjunction with FIG. 3 , a beauty product 224 may include theinformation recited above.

The method 3300 optionally includes where the information regarding thebeauty product comprises an indication of first effects of applying thebeauty product to the body part. For example, beauty product data 302may include effects 334, which indicate the effects 334 of applying thebeauty product 224 with beauty product data 302.

The method 3300 optionally includes determining second effectsindicating changes to the video data from the presenter applying thebeauty product to the body part of the presenter, wherein the secondeffects are determined based on the indication of the first effects ofapplying the beauty product to the body part and the video data. Forexample, as disclosed in conjunction with FIG. 1 , determine effectsmodule 128 is configured to determine the tutorial effects 218 of FIGS.2, 4, 5A, 5B, 5C, 8, 11, 12, 13, and 14 from AR tutorial video 228.

The method 3300 optionally includes where the video data is first videodata, and where the method further includes determining third effectsindicating changes to make to second video data of a user to simulateapplying the beauty product to the body part of the user, where thethird effects are determined based on the second effects. For example,as disclosed in conjunction with FIG. 1 , determine effects module 128is configured to determine the AR effects 219 of FIGS. 2, 4, 5A, 5B, 5C,8, 11, 12, 13, and 14 from tutorial effects 218 and/or AR tutorial video228. FIGS. 5A, 5B, and 5C illustrate an example of adding AR effects219A, 219B, 219C, to a body part 222 that is a right eye of a user 238from tutorial effects 218A, which are the changes to a right eye of apresenter 236 from applying beauty product 224B with beauty product224A. FIGS. 5A, 5B, and 5C illustrate the progression of the applicationof the eye shadow where the changes to the right eye of the user 238with the AR effects 219A, 219B, and 219C are synchronized with thechanges to the right eye of the presenter 236, which are indicated bytutorial effects 218A, 218B, and 218C. The method 3300 may furtherinclude storing, within the data store, second association dataassociating the third effects with the beauty product and the tutorial.For example, AR effects 219 may be stored as part of AR tutorial data202.

The method 3300 optionally includes where the indication of the firsteffects of applying the beauty product to the body part comprises one ormore of the following group: a color, a texture, a transparency, awidth, a length, a suggested skin type for the body part, and asuggested hair type for the body part. For example, effects 334 of FIG.3 may include the first effects listed above.

The method 3300 optionally includes where the beauty product is a firstbeauty product and where processing the video data includes receivinginput from the presenter that indicates selection of a second beautyproduct displayed on the display device and identifying the first beautyproduct as the second beauty product. For example, as disclosed inconjunction with FIGS. 1-3, 18, 28-30, and 33-45 . In one example, asillustrated in FIG. 29 , presenter 236 selects with her finger 810C abeauty product 224.

The method 3300 optionally includes recording the video data using acomputing device, during the recording of the video data, receiving,input from the presenter to identify the beauty product, and focusing adisplay, on the computing device, of the video data on images of thebeauty product. For example, as disclosed in conjunction with FIGS. 1-3,18, 28-30, and 33-45 . In one example, as illustrated in FIG. 41 , thepresenter 236 may invoke scan beauty product 4112 from screens 4100A and4100B of FIG. 41 . Screen 4200A of FIG. 42 may be displayed in responseto the invocation of scan beauty product 4112, where scan window 4202focuses on live images of the beauty product 224.

The method 3300 optionally includes where the beauty product is a firstbeauty product, and the processing of the video data to identify thebeauty product includes matching image data of the first beauty productin the images to a stored image data of a second beauty product andidentifying the first beauty product as the second beauty product. Forexample, as disclosed in conjunction with FIGS. 1-3, 18, 28-30, and33-45 . In one example, match name module 3404 or match image module3402 may match an image of the beauty product as displayed on screen3600B of FIG. 36 or screen 4200A of FIG. 42 with stored images asdisclosed in conjunction with FIG. 34 .

The method 3300 optionally includes identifying a product code from animage of the beauty product, matching the product code to stored productcodes to identify the beauty product, and retrieving informationregarding the beauty product using the product code, wherein the productcode is an identifier assigned to the beauty product. For example, asdisclosed in conjunction with FIGS. 1-3, 18, 28-30, and 33-45 . In oneexample, identify produce code module 3408 identifies a product code3610 from screen 3600B or screen 4200A and matches the product code 3610to a stored product code as disclosed in conjunction with FIG. 34 .

The method 3300 optionally includes where the video data is live videodata, and where the method further includes in response to determiningthat the beauty product is within the live video data, causing aninterrogation signal to be transmitted to the beauty product to solicita radio frequency identification (RFID) identifying the beauty product,receiving radio signals comprising the RFID from the beauty product, theRFID comprising a product code, and identifying the beauty product usingthe product code. For example, as disclosed in conjunction with FIGS.1-3, 18, 28-30, and 33-45 . In one example as disclosed in conjunctionwith FIG. 45 , RFID reader 4502 may receive RFID 4506 and identify RFIDmodule 3406 may identify the beauty product 224 based on the RFID 4506signal.

The method 3300 optionally includes generating an identifier for thebeauty product, the identifier identifying the beauty product to anapplication recording the tutorial, after the causing presentation ofinformation regarding the beauty product on the display device,receiving, input from the presenter, the input comprising custominformation regarding the beauty product, storing a generic identifierof the beauty product and the custom information based on the generatedidentifier. For example, as disclosed in conjunction with FIG. 34determine beauty product module 126 may generate an identifier forbeauty product data 302. In another example, the presenter 236 may entercustom information in screen 3700 and screen 3800 and determine beautyproduct module 126 may store the information as beauty product data 302with a generated identifier in a local or remote data store such as adatabase.

The method 3300 optionally includes where the information regarding thebeauty product comprises an identity of the beauty product and where themethod further includes causing presentation of a user interface itemfor the presenter to confirm or reject the identity of the beautyproduct, and responding to receiving, via an application implemented byat least one processor, input from the presenter that indicatesrejection of the identity of the beauty product, by recording new imagesof the presenter creating the tutorial, and identifying the beautyproduct based on the new images. For example, as disclosed inconjunction with FIG. 42 , a presenter 236 may select scan again 4210 toreturn to screen 4200A. AR tutorial 143 may then capture new images ofthe beauty product 224 to try and identify the beauty product 224.

The method 3300 optionally includes where the beauty product isassociated with a step of the tutorial, where the tutorial comprises aplurality of steps, and where the method further includes causingpresentation of information regarding each step of the plurality ofsteps on the display device and in response to receiving an indicationof a selection of a step of the plurality of steps, associating thebeauty product with the step. For example, as disclosed in conjunctionwith FIGS. 1-3, 18, 28-30, and 33-45 . For example, a screen similar toscreen 4000 may be displayed that enables the presenter 236 to select anadd 4006B or add 4006C to add an identified beauty product 224 to stepof the AR tutorial.

The method 3300 optionally includes in response to a selection by thepresenter, displaying a preview of a plurality of tutorials to thepresenter, wherein each of the plurality of tutorials use the beautyproduct. For example, as disclosed in conjunction with FIG. 43 , ARtutorial A description 4304A, AR tutorial B description 4304B, and ARtutorial C description 4304C are displayed on screen 4300A as example ARtutorials that include beauty product 224. One or more of the operationsof method 3300 may be optional. Method 3300 may include one or moreadditional operations. The operations may be performed in a differentorder.

FIG. 34 illustrates components of determine beauty product module 126,in accordance with some embodiments. Determine beauty product module 126is configured to determine an identity of a beauty product 224. Asdisclosed in conjunction with FIGS. 1-3 , beauty product 224 is a beautyproduct that is associated with beauty product data 302. In someembodiments, determine beauty product module 126 determines the beautyproduct 224 via UI screens presented to the presenter 236. For example,as illustrated in FIG. 29 presenter 236 selects beauty product 224 bymaking selections from edit menu 2402 and beauty product list 2908A orbeauty product list 2908B.

In some embodiments, determine beauty product module 126 is configuredto take an image of a beauty product 326 and determine the beautyproduct 224 based on a product code 330 on the beauty product 224 andbased on identifying the beauty product 224 from a database of beautyproducts 224. For example, the determine beauty product module 126 usesa trained neural network to perform object recognition of the beautyproduct 224 so that the presenter 236 does not have to enter informationregarding the beauty product 224. In some embodiments, determine beautyproduct module 126 retrieves beauty product data 302 from a local orremote database, such as beauty products 2018 of FIG. 20 . Images ofbeauty product 326 may be used to request confirmation from thepresenter 236 and/or to display the beauty product 224, such as in FIG.4 where two beauty products 224 are displayed as beauty product 224A andbeauty product 224B. Determine beauty product module 126 is configuredto determine the beauty product 224 as disclosed in conjunction withFIGS. 29, 30, and 35-46 , in accordance with some embodiments.

In some embodiments, determine beauty product module 126 generatesbeauty product data 302 for identified beauty products and generates anidentifier for the beauty product data 302. Beauty product data 302 maybe stored in a database such as beauty product 2018 of FIG. 20 with anidentifier generated by determine beauty product module 126. Determinebeauty product module 126 may display information related to a beautyproduct 224 by retrieving a record of beauty product data 302 frombeauty product 2018 of FIG. 20 . Determine beauty product module 126 maythen use information in beauty product information 304 to access amanufacturer website to retrieve additional information regarding thebeauty product 224. In some embodiments, the information in beautyproduct data 302 that was received by the presenter 236 or added by theAR tutorial 143 is considered customized information or data and theinformation from the website of the manufacturer may be termedmanufacturer information.

The match image module 3402, the match name module 3404, the identifyRFID module 3406, and/or the identify product code module 3408 areconfigured to identify changes to the body part of the presenter from anapplication of a beauty product 224 and identify the beauty product 224based on the changes to the body part 222 of the presenter 236. Forexample, determine effects module 128 is configured to determinetutorial effects 218 as disclosed herein. The match image module 3402,the match name module 3404, the identify RFID module 3406, and/or theidentify product code module 3408 may use the tutorial effects 218 todetermine a color 316 of the beauty product 224 and use the color 316 toassist in identifying the beauty product 224. Other characteristics ofthe tutorial effects 218 may be used to identify the beauty product 224.

The match image module 3402, the match name module 3404, and theidentify product code module 3408 are further disclosed in conjunctionwith FIG. 35-45 and, in particular, FIG. 36 . The identify radiofrequency identification (RFID) module 3406 is further disclosed inconjunction with FIGS. 35-45 , and, in particularly, FIG. 45 .

FIG. 35 illustrates components of adding beauty products to augmentedreality tutorials 3500, in accordance with some embodiments. Camera 130,screen 132, AR tutorial 143, and mobile device 146 are the same orsimilar as elements with like numbers and/or names in FIG. 1 . Point ofentry state 3502 is a place in an application that invokes record ARtutorial 143. In some embodiments, point of entry state 3502, add beautyproducts state 3510, record AR tutorial state 3522, view AR tutorialstate 3516, add beauty products state 3520, and outside AR tutorialstate 3524 are states that the presenter 236 may be in within one ormore applications. In some embodiments, a state indicates commands thatmay be invoked by the user of the application or commands that theapplication may invoke based on actions of the user.

The enter 3504, exit 3506, add beauty product 3508, return to recordingor editing 3512, complete recording AR tutorial 3514, and add beautyproduct 3518 are commands 112 as described in conjunction with FIG. 1 ,in accordance with some embodiments.

For example, the presenter 236 may be within a communication applicationand may select “AR tutorial A description” 4304A of FIG. 43 to invokethe enter 3504 command and be at record AR tutorial state 3522. Apresenter 236 may record an AR tutorial 143 at record AR tutorial state3522 and invoke add beauty products 3508 command to enter add beautyproducts state 3510. FIG. 29 illustrates an example of add beautyproducts 3508 where the presenter 236 selects add beauty product icon2902 to add a beauty product 224. FIG. 30 illustrates another example ofadd beauty products 3508 where the presenter 236 selects add beautyproduct icon 3002 to invoke add beauty product 2904 command. FIG. 36illustrates another example of add beauty products 3508 where thepresenter 236 selects add beauty product icon 3602 to invoke add beautyproduct 3604 command. FIG. 40 illustrates another example of add beautyproducts 3508 where the presenter 236 selects add 4006B, 4006C to addbeauty products 224. FIG. 41 illustrates another example of add beautyproducts 3508 where the presenter 236 selects add beauty product icon4106 to invoke scan beauty product 4108 command. FIG. 45 illustratesanother example of add beauty products 3508 where the mobile device 146triggers by an interrogation signal 4504 signal a RFID 4506 thatidentifies the beauty product 224.

The presenter 236 may invoke return to recording or editing 3512 commandfrom add beauty products state 3510. For example, in FIG. 29 , presenter236 may complete the addition of the beauty product 224 to step 2 1608Bof the AR tutorial 143, which returns the presenter 236 to record ARtutorial state 3522. Additionally, in FIGS. 29, 30, 36, 40, 41, and 45the presenter 236 may return to record AR tutorial state 3522 after abeauty product 224 is identified.

The complete recording AR tutorial 3514 command may be invoked by thepresenter 236 when they are done recording the AR tutorial 143. Thepresenter 236 may enter view AR tutorial state 3516. For example, inFIG. 40 , the presenter 236 may be taken to screen 4000 where the step 1information 4002A, step 2 information 4002B, and step 2 information4002C are illustrated with the associated beauty product 224.

The presenter 236 may invoke add beauty product 3518 command from viewAR tutorial state 3516 to be brought to add beauty products state 3520.For example, in FIG. 40 , the presenter 236 may select add 4006B or add4006C to invoke the add beauty product 3518 command to be brought to theadd beauty products state 3520. Add beauty products state 3520 mayprovide the same or similar options for the presenter 236 to add abeauty product 224 to the AR tutorial 143 as add beauty products state3510. The presenter 236 may return from add beauty products state 3520to the point of entry state 3502, the view AR tutorial state 3516, orthe record AR tutorial state 3522, in accordance with some embodiments.In some embodiments, the presenter 236 is in the outside AR tutorialstate 3524 and enters add beauty products state 3520. For example, asillustrated in FIG. 41 , the presenter 236 may not be within the ARtutorial 143, but selects add beauty product icon 4104, which invokesscan beauty product 4108 that enables the presenter 236 to add a beautyproduct 224 to the AR tutorial 143.

FIG. 35 illustrates the operation of one or more applications related toadding beauty products 224 to an AR tutorial 143. In some embodiments,the states and/or commands 112 are organized differently. In someembodiments, there are fewer or a greater number of states and/orcommands 112.

FIG. 36 illustrates components of adding beauty products to ARtutorials, in accordance with some embodiments. Camera 130, mobiledevice 146, and AR tutorial 143 are the same or similar as elements withlike numbers in FIG. 1 . Screen 3600A and screen 3600B are the same orsimilar as screen 132. Beauty product 224, step title 248, and presenter236 are the same or similar as elements of like numbers of FIG. 2 . Stepmark 426, play bar 422, current play 424, play bar 422, and back 428 arethe same or similar as elements with like numbers of FIG. 4 . Finger 810is the same or similar as finger 810 of FIG. 8 . Edit and enhance 1408may be the same or similar as edit and enhance 1408 of FIG. 14 . Step 21608B is the same or similar as step 2 1608B of FIG. 16 . Edit menu 2402is the same or similar as edit menu 2402 of FIG. 24 . Back 428 and back3605 offer the presenter 236 the option to return to a previous screensuch as screen 3600A.

As illustrated, presenter 236 of screen 3600A invokes command 112 addbeauty product 3604 by selecting add beauty product icon 3602 with herfinger 810A. In response, determine beauty product module 126 displaysscreen 3600B where the presenter 236 may hold the beauty product 224. Insome embodiments, determine beauty product module 126 displays a steptitle 248 associated with the step indicated by current play 424, whichin this case is step 2 1608B. The presenter 236 may capture 3606 animage of beauty product 224 and edit and enhance 1408 the image beforecapturing the image. In some embodiments capture 3606 assists thepresenter 236 in displaying the beauty product 224. For example, capture3606 may become a bright color when a beauty product 224 is identifiedor is becoming closer to being identified. Determine beauty productmodule 126 may provide instructions on the screen 3600B such as “turnproduct”, “produce name not visible”, “product code not visible”,“product identified”, “product not identified yet”, and so forth. Insome embodiments, determine beauty product module 126 may put an arrowon the screen 3600B or a similar image to instruct the presenter 236 totwist or turn the beauty product 224. In some embodiments, determinebeauty product module 126 identifies the beauty product 224 based onproduct packaging 3608, product name 3612, and/or produce code 3610.

The product packaging 3608 indicates the image of the product packagingfrom the presenter 236 holding the beauty product 224 in front of thecamera 130. The image displayed on screen 3600B may be from a previouslycaptured image that the presenter 236 has selected to be identified as abeauty product 224, such as for inclusion in the AR tutorial 143. Thematch image module 3402 may match product packaging 3608 to productpackaging images from a database. For example, beauty products 2018 ofFIG. 20 may include images of beauty products 326 as disclosed inconjunction with FIG. 3 . The match image module 3402 may be a neuralnetwork trained with deep learning using images from beauty products2018 of FIG. 20 . The match image module 3402 may perform matching in adifferent way.

The match name module 3404 may perform optical character recognition toidentify the product name 3612. The product name 3612 may be part ofbeauty product information 304 of FIG. 3 . The match name module 3404may include a neural network trained with deep learning to identify theproduct name 3612 or other information displayed in the image of thebeauty product 224 such as a manufacturer, trademark or other such itemsincluded on the product packaging 3608. The match name module 3404 mayaccess a database of beauty products 224 such as beauty product 2018 ofFIG. 20 to identify a manufacture, trademark, product name 3612 or othersuch identifying items on the product packaging 3608. The match namemodule 3404 may make a partial match such as manufacturer and displaythe manufacturer and offer options to the presenter 236 to continue theidentification such as to go to the manufacturer website to identify thebeauty product 224.

The identify product code module 3408 may identify the product code3610. The identify product code module 3408 may include a neural networktrained with deep learning to identify the product code 3610. Theidentify product code module 3408 may access a database of beautyproducts 224 such as beauty product 2018 of FIG. 20 to identify thebeauty product 224 from a database of product codes 3610. The productcode 3610 may be the same or similar as product codes 330 of FIG. 3 .The product code 3610 may be a bar code or other similar code. Theidentify radio frequency identification (RFID) module 3406 is disclosedin conjunction with FIG. 45 .

The match image module 3402, match name module 3404, identify RFIDmodule 3406, and/or identify product code module 3408 are configured topresent a list of beauty products 224 to the presenter 236 to chose fromwhen the identification results in multiple beauty products 224 beingpossible matches, in accordance with some embodiments.

The match image module 3402, match name module 3404, identify RFIDmodule 3406, and/or identify product code module 3408 are configured toidentify the beauty product 224 based on a body part 222 associated witha step n 204 of the AR tutorial 143, a body part 308 associated with thebeauty product 224, and/or a manufacturer of the beauty product 224.

FIG. 37 illustrates information regarding a beauty product, inaccordance with some embodiments. The information regarding the beautyproduct may be displayed to presenter 236 and/or user 238 as illustratedon screen 3700. The presenter 236 and/or user 238 may edit one or morefields of the information regarding the beauty product.

The information regarding the beauty product includes AR tutorial name3702, step 3704, beauty product name 3706, body part 3708, URL ofpurchase page 3710, manufacturer name 3714, product codes 3716, locationof variations 3718, location of image 3720, location of thumbnail 3722,location of ratings 3724, ingredients 3726, and competing products 3728.The information regarding beauty product may be the same or similar aselements disclosed in conjunction with AR tutorial data 202 of FIG. 2and/or beauty product data 302 of FIG. 3 .

A selection of the icon “images” or “competing products” brings thepresenter 236 to a user interface to select images from a local orremote location such as storage of the mobile device 146, the internet,or from a database of beauty products 224 such as beauty product 2018 ofFIG. 20 . A selection of the icon “OK” may accept changes and/or invokea command 112 to return to a previous screen. A selection of back 428may reject changes and return to a previous screen. One or more of theinformation field may not be displayed in information regarding a beautyproduct. One or more additional information fields may be included ininformation regarding a beauty product. In some embodiments theinformation is presented differently than as illustrated in screen 3700.Screen 3700 may be arrived at after a beauty product 224 is identifiedor when a beauty product 224 is edited, in accordance with someembodiments.

FIG. 38 illustrates information regarding a beauty product, inaccordance with some embodiments. The information regarding the beautyproduct may be displayed to presenter 236 and/or user 238 on screen3800. The presenter 236 and/or user 238 may edit one or more fields ofthe information regarding the beauty product.

The information regarding the beauty product or associated with thebeauty product includes AR tutorial name 3802, step 3804, beauty productname 3806, body part 3808, URL of purchase page 3810, upload new image3812, confirm 3814, attach to application account 3816, attach to beautytutorial 3818, image of beauty product 3820, and more details 3822. Theinformation regarding beauty product may be the same or similar aselements disclosed in conjunction with AR tutorial data 202 of FIG. 2and/or beauty product data 302 of FIG. 3 .

A selection of upload new image 3812 brings the presenter 236 to an userinterface to select images from a local or remote location such asstorage of the mobile device 146, the internet, or from a database ofbeauty products 224 such as beauty product 2018 of FIG. 20 . A selectionof “confirm” may save changes to information regarding the beautyproduct. A selection of “attach to application account” provides thepresenter 236 with the option of attaching the beauty product 224associated with information regarding the beauty product to a useraccount of an application such as a communication application asdisclosed herein. A selection of “attach to beauty tutorial” gives thepresenter 236 the option of attaching the beauty product 224 withinformation regarding the beauty product to an AR tutorial such as ARtutorial data 202. Image of beauty product 3820 may be a current imageof the beauty product 3820 stored in beauty product data 302. Selectionof “more details” 3822 brings the presenter 236 to a screen where moreinformation regarding the beauty product 224 displayed with the optionto edit the information. A selection of back 428 may cancel any changesand return the presenter 236 to a difference screen.

One or more of the information fields may not be displayed ininformation regarding a beauty product. One or more additionalinformation fields may be included in information regarding a beautyproduct. In some embodiments the information is presented differentlythan as illustrated in screen. Screen 3800 may be arrived at after abeauty product 224 is identified or when a beauty product 224 is edited,in accordance with some embodiments.

FIG. 39 illustrates components of adding beauty products to augmentedreality tutorials, in accordance with some embodiments. The componentsof adding beauty products may be displayed to presenter 236 and/or user.The presenter 236 and/or user may edit one or more fields of theinformation regarding the beauty product displayed on screen 3700.Screen 3900 includes AR tutorial name 3902, step 3904, enter search ofinternet 3906, search your beauty items 3908, URL of purchase page 3910,suggestions which include beauty product 1 3912A, beauty product 23912B, and beauty product 3 3912C, and dropdown menu 3914.

The presenter 236 and/or user may select a beauty product 224 for a step3904 of an AR tutorial name 3902 via screen 3900. The presenter 236 mayrequest suggestions by setting the dropdown menu that indicates “bodypart” 3914. The dropdown menu may have other options such as “popular”,“favorites of celebrity XYZ”, “suggestions based on your preferences”,and so forth. A selection of back 428 cancels any changes and returns toa previous screen, in accordance with some embodiments. In someembodiments a screen is displayed to query the presenter 236 or user tokeep or reject changes. The beauty product 224 that is identified inscreen 3900 is attached to AR tutorial name 3902 such as AR tutorialdata 202 and step N 204. Screen 3900 may bring the presenter 236 or userto screen 3700 or screen 3800 so that the presenter 236 or user may editinformation regarding the beauty product 224. Screen 3900 may be arrivedafter a beauty product 224 is identified or when a beauty product 224 isedited, in accordance with some embodiments. Screen 3900 may be arrivedat when a beauty product 224 cannot be identified or the presenter 236or user requests help in finding a beauty product 224.

One or more of the information fields may not be displayed ininformation regarding a beauty product 224. One or more additionalinformation fields may be included in information regarding a beautyproduct. The information is presented differently than as illustrated inscreen 3900. Screen 3900 may be arrived at after a beauty product 224 isidentified or when a beauty product 224 is edited, in accordance withsome embodiments. Screen 3900 may correspond to add beauty productsstate 3510 of FIG. 35 or add beauty products state 3520.

FIG. 40 illustrates components of adding beauty products to ARtutorials, in accordance with some embodiments. Camera 130, mobiledevice 146, and AR tutorial 143 are the same or similar as elements withlike numbers in FIG. 1 . Beauty product 224 is the same or similar asbeauty product 224 of FIG. 2 . Body part 308 is the same or similar asbody part 308 of FIG. 3 . Finger 810 is the same or similar as finger810 of FIG. 8 . Back 428 may be selected to return to a previous screen.Back 428 may cancel changes and/or the changes may be confirmed.

Illustrated in screen 4000 is step 1 information 4002A, step 2information 4002B, step 3 information 4002C, beauty product 224, bodypart 308, more 4004, add 4006B, and add 4006C. Screen 4000 may bereached after or during an AR tutorial 143 is being recorded by apresenter 236. Screen 4000 illustrates the different beauty products 224associated with the different steps of the AR tutorial 143. Thepresenter 236 may add 4006B or add 4006C a beauty product 224. Forexample, selection of add 4006B or add 4006C may bring the presenter 236to screen 3900. A selection of more 4004 may bring the presenter 236 toscreen 3700 or screen 3800, in accordance with some embodiments. Screen4000 enables the presenter 236 to manage the beauty products 224associated with AR tutorial 143. Screen 4000 may correspond to addbeauty products state 3510 of FIG. 35 or add beauty products state 3520.

One or more of the information fields may not be displayed in screen4000. One or more additional information fields may be included inscreen 4000. In some embodiments the information is presenteddifferently than as illustrated in screen 4000. Screen 4000 may bearrived after a beauty product 224 is identified, when a beauty product224 is edited, when a step of the AR tutorial 143 is completed, or whenthe AR tutorial 143 is completed, in accordance with some embodiments.

FIG. 41 illustrates components of adding beauty products to ARtutorials, in accordance with some embodiments. Camera 130, mobiledevice 146, and AR tutorial 143 are the same or similar as elements withlike numbers and/or names as disclosed in conjunction with FIG. 1 .Beauty product 224 is the same or similar as beauty product 224 of FIG.2 . Back 428 is the same or similar as back 428 as disclosed inconjunction with FIG. 4 . Back 4102 is the same or similar as back 428.Menu 4104 provides enables the presenter 236 to invoke commands 112. Asillustrated in screen 4100A, beauty product 224 is being held by a handof the presenter 236 in front of the camera 130 where screen 4100Aillustrates live images from the camera 130.

As illustrated, the presenter 236 selects add beauty product icon 4106to invoke scan beauty product 4108 command. Scan beauty product 4108 isa command 112 as disclosed in conjunction with FIG. 1 . In response tothe invocation of scan beauty product 4108, screen 4100B illustratesscan 4109 and images 4110. A selection of images 4110 provides thepresenter 236 with options for identifying the beauty product 224 asdisclosed herein or selecting an image from a stored image as disclosedherein to have a beauty product 224 captured in the stored imageidentified. A selection of scan 4109 invokes scan beauty product 4112command, which is a command 112 as disclosed in conjunction with FIG. 1. Back 428 may cancel and/or confirm and bring the presenter 236 to aprevious screen.

One or more of the information fields may not be displayed in screen4100A or screen 4100B. One or more additional information fields may beincluded in screen 4100A or screen 4100B. In some embodiments theinformation is presented differently than as illustrated in screen 4100Aor screen 4100B. Screen 4100A and screen 4100B may be arrived at when astep of the AR tutorial 143 is completed, when the AR tutorial 143 iscompleted, or when the AR tutorial 143 or presenter 236 would like toidentify a beauty product 224, in accordance with some embodiments.

FIG. 42 illustrates components of scanning a beauty product, inaccordance with some embodiments. Camera 130, mobile device 146, and ARtutorial 143 are the same or similar as elements with like numbersand/or names as disclosed in conjunction with FIG. 1 . Beauty product224 and presenter 236 are the same or similar as like named and/ornumbered elements as disclosed in conjunction with FIG. 2 . “Attach toapplication Account” 3816 and “Attach to AR tutorial” 3818 are the sameor similar as like named and/or numbered elements as disclosed inconjunction with FIG. 38 . Back 428 may be selected to return to aprevious screen.

As illustrated in screen 4200A, beauty product 224 is being held by ahand of the presenter 236 in front of the camera 130 where screen 4200Aillustrates live images from the camera 130. A scan window 4202 isdisplayed to indicate where determine beauty product module 126 isfocused on scanning to look for product packaging 3608, product name3612, and-or product code 3610 as disclosed in conjunction with FIG. 36. The presenter 236 may select a portion of the beauty product 224 forthe determine beauty product module 126 to focus on such as a productcode 3610 or product name 3612 as disclosed in conjunction with FIG. 36. When the determine beauty product module 126 identifies the beautyproduct 224 it may invoke identified beauty product 4204, which is acommand 112 as disclosed in conjunction with FIG. 1 . Screen 4200B maybe displayed which includes information of beauty product 4206 such asone or more fields of screen 3700, screen 3800, or other information.The check 4205 indicates the beauty product 224 has been identified. Thepresenter 236 may “attach to application account” 3816 or “attach to ARtutorial” 3818 as disclosed in conjunction with FIG. 38 . The presenter236 may indicate that the beauty product 224 was incorrectly identifiedby selection of “scan again” 4210.

One or more of the information fields may not be displayed in screens4200A, 4200B. One or more additional information fields may be includedin screens 4200A, 4200B. In some embodiments the information ispresented differently than as illustrated in screens 4200A, 4200B.Screen 4200B may go to screen 3700 or screen 3800 to provide additionalinformation or enable the presenter 236 to add information associatedwith the beauty product 224. In some embodiments, one or more of thecommands 112 may be different or may not be present. In some embodimentsdifferent commands 112 invoke the presentation of the screens 4200A,4200B.

FIG. 43 illustrates previewing an AR tutorial, in accordance with someembodiments. Mobile device 146 is the same or similar as mobile device146 as disclosed in conjunction with FIG. 1 . Beauty product 224 is thesame or similar as beauty product 224 as disclosed in conjunction withFIG. 2 . Screen 4300A and screen 4300B are the same or similar as screen132. Screen 4300A illustrates beauty product 224 where “beauty productinformation” indicates information regarding the beauty product 224 isdisplayed such as is disclosed in conjunction with FIGS. 3, 37, and 38 .AR tutorial A description 4304A, AR tutorial B description 4304B, and ARtutorial C description 4304C indicate AR tutorials that include beautyproduct 224. AR tutorial A description 4304A, AR tutorial B description4304B, and AR tutorial C description 4304C may include a video previewand other information regarding the AR tutorial such as is disclosed inconjunction with FIG. 2 and herein. The see more 4306 may be selected toinvoke a command 112 to see additional AR tutorials that include thebeauty product 224. The similar products 4308 icon may be selected tosee AR tutorials with competing products 328 to beauty product 224.

Beauty product 224A and beauty product 224B indicate different beautyproducts 224 that may be selected to explore AR tutorials related tobeauty product 224A and beauty product 224B. Screen 4300A enables apresenter 236 and/or user to explore AR tutorials based on the beautyproducts 224 that are used in the AR tutorials, in accordance with someembodiments.

Selecting the AR tutorial A description 4304A or another AR tutorialicon invokes preview of AR tutorial A 4310, which is a command 112.Selecting AR tutorial A description 4304A brings the user or presenter236 to screen 4300B to preview of AR tutorial A 4310. Screen 4300Bprovides a preview of AR tutorial A 4310 as described herein. Asillustrates AR tutorial A is presented in a full screen mode of the ARtutorial display 142 where AR tutorial display 142 is disclosed inconjunction with FIG. 1 .

A selection of view product 4312 by the presenter 236 or user 238invokes the command view products 4314, which is a command 112 asdisclosed herein. Invocation of view products 4314 may bring thepresenter 236 or user to screen 4400 or another screen to view thebeauty products of AR tutorial A.

One or more of the information fields may not be displayed in screen4300A or screen 4300B. One or more additional information fields may beincluded in screen 4300A or screen 4300B. In some embodiments theinformation is presented differently than as illustrated in screen 4300Aor screen 4300B. In some embodiments one or more of the commands 112 maybe different or may not be present. In some embodiments differentcommands 112 invoke the presentation of the screen 4300A or screen4300B.

FIG. 44 illustrates displaying beauty products associated with an ARtutorial, in accordance with some embodiments. Mobile device 146 is thesame or similar as mobile device 146 as disclosed in conjunction withFIG. 1 . Beauty product 224 is the same or similar as beauty product 224as disclosed in conjunction with FIG. 2 . Screen 4400 is the same orsimilar as screen 132. Screen 4400 illustrates beauty products 224A,224B, 224C, and 224D associated with an AR tutorial such as AR tutorial143. Screen 4400 may be presented to the presenter 236 or user 238 inresponse to an invocation of view products 4314. A selection of one ofbeauty products 224A, 224B, 224C, and 224D may cause screen 3700 orscreen 3800 to be displayed, in accordance with some embodiments. Title4402 indicates a title for the screen 4400 of “Beauty ProductsAssociated with AR Tutorial.”

One or more of the information fields may not be displayed in screen4400. One or more additional information fields may be included inscreen 4400. In some embodiments the information is presenteddifferently than as illustrated in screen 4400. In some embodiments oneor more of the commands 112 may be different or may not be present. Insome embodiments different commands 112 invoke the presentation of thescreen 4400.

FIG. 45 illustrates components of scanning a beauty product 4500, inaccordance with some embodiments. Camera 130, screen 132, and mobiledevice 146 are the same or similar as like numbered or named elements asdisclosed in conjunction with FIG. 1 . Beauty product 224 is the same orsimilar as beauty product 224 as disclosed in conjunction with FIG. 2 .RFID reader 4502 is a radio transmitter and radio receiver, inaccordance with some embodiments.

Identify RFID module 3406 of FIG. 34 is configured to cause RFID reader4502 to send interrogation signal 4504 and receive RFID 4506 where RFID4506 includes beauty product identification encoded in radio waves.Interrogation signal 4504 may be electromagnetic interrogation pulsesthat indicate that RFID tag 4508 should respond with RFID 4506.Interrogation signal 4504 solicits the RFID tag 4508 to respond with theRFID 4506. RFID tag 4508 comprises a radio transponder, a radioreceiver, and transmitter. In some embodiments RFID tag 4508 comprises abattery. In some embodiments RFID tag 4508 does not include a batteryand may use energy captured from the interrogation signal 4504 totransmit RFID 4506.

In some embodiments when the beauty product 224 is identified thenscreen 3700, screen 3800, or another screen with additional informationregarding the beauty product 224 is displayed for the presenter 236 oruser 238. In some embodiments identify RFID module 3406 may sendinterrogation signal 4504 based on a detection of a beauty product 224within an AR tutorial 143 of FIG. 1 or AR tutorial 606 of FIG. 6 . In anexample, when a presenter 236 selects add beauty product icon 3602,identify RFID module 3406 sends interrogation signal 4504. Identify RFIDmodule 3406 may present a screen to confirm that the identified beautyproduct 224 is the correct one or to instruct the presenter 236 to movethe beauty product 224 closer to the mobile device 146. In anotherexample, identify RFID module 3406 may send interrogation signal 4504when a beauty product 224 is detected in screen 3000B of FIG. 30 .Identify RFID module 3406 may determine whether to send interrogationsignal 4504 based on whether a current step 1608B has an associatedbeauty product 224, in accordance with some embodiments. In someembodiments, mobile device 146 includes transceiver circuitry,processing circuitry, and one or more antennas configured to operateRFID reader 4502.

Some embodiments provide a technical solution to enabling a tutorial tobe presented to a user with a live-view image of the user having AReffects added where the AR effects mirror the actions of a presenter ofthe tutorial. Some embodiments provide a technical solution tosynchronizing motion and make-up effects of a prerecorded video withlive view images of a user of a tutorial where the motion and make-upeffects are added to the live view images. For example, as disclosed inconjunction with FIG. 1 , generate user images module 108 is configuredto display AR motion 217 without the AR effects 219 so that the user 238can apply the beauty product 224 on the user themselves. Someembodiments provide a technical solution to digitally representing abeauty product displayed within a live stream video during a recordingof an AR tutorial.

The following discussion relates to various terms or phrases that arementioned throughout the subject disclosure. “Signal Medium” refers toany intangible medium that is capable of storing, encoding, or carryingthe instructions for execution by a machine and includes digital oranalog communications signals or other intangible media to facilitatecommunication of software or data. The term “signal medium” shall betaken to include any form of a modulated data signal, carrier wave, andso forth. The term “modulated data signal” means a signal that has oneor more of its characteristics set or changed in such a matter as toencode information in the signal. The terms “transmission medium” and“signal medium” mean the same thing and may be used interchangeably inthis disclosure.

“Communication Network” refers to one or more portions of a network thatmay be an ad hoc network, an intranet, an extranet, a virtual privatenetwork (VPN), a local area network (LAN), a wireless LAN (WLAN), a widearea network (WAN), a wireless WAN (WWAN), a metropolitan area network(MAN), the Internet, a portion of the Internet, a portion of the PublicSwitched Telephone Network (PSTN), a plain old telephone service (POTS)network, a cellular telephone network, a wireless network, a Wi-Fi®network, another type of network, or a combination of two or more suchnetworks. For example, a network or a portion of a network may include awireless or cellular network and the coupling may be a Code DivisionMultiple Access (CDMA) connection, a Global System for Mobilecommunications (GSM) connection, or other types of cellular or wirelesscoupling. In this example, the coupling may implement any of a varietyof types of data transfer technology, such as Single Carrier RadioTransmission Technology (1×RTT), Evolution-Data Optimized (EVDO)technology, General Packet Radio Service (GPRS) technology, EnhancedData rates for GSM Evolution (EDGE) technology, third GenerationPartnership Project (3GPP) including 3G, fourth generation wireless (4G)networks, Universal Mobile Telecommunications System (UMTS), High SpeedPacket Access (HSPA), Worldwide Interoperability for Microwave Access(WiMAX), Long Term Evolution (LTE) standard, others defined by variousstandard-setting organizations, other long-range protocols, or otherdata transfer technology.

“Processor” refers to any circuit or virtual circuit (a physical circuitemulated by logic executing on an actual processor) that manipulatesdata values according to control signals (e.g., “commands”, “op codes”,“machine code”, etc.) and which produces corresponding output signalsthat are applied to operate a machine. A processor may, for example, bea Central Processing Unit (CPU), a Reduced Instruction Set Computing(RISC) processor, a Complex Instruction Set Computing (CISC) processor,a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), anApplication Specific Integrated Circuit (ASIC), a Radio-FrequencyIntegrated Circuit (RFIC) or any combination thereof. A processor mayfurther be a multi-core processor having two or more independentprocessors (sometimes referred to as “cores”) that may executeinstructions contemporaneously.

“Machine-Storage Medium” refers to a single or multiple storage devicesand/or media (e.g., a centralized or distributed database, and/orassociated caches and servers) that store executable instructions,routines and/or data. The term shall accordingly be taken to include,but not be limited to, solid-state memories, and optical and magneticmedia, including memory internal or external to processors. Specificexamples of machine-storage media, computer-storage media and/ordevice-storage media include non-volatile memory, including by way ofexample semiconductor memory devices, e.g., erasable programmableread-only memory (EPROM), electrically erasable programmable read-onlymemory (EEPROM), FPGA, and flash memory devices; magnetic disks such asinternal hard disks and removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks The terms “machine-storage medium,”“device-storage medium,” “computer-storage medium” mean the same thingand may be used interchangeably in this disclosure. The terms“machine-storage media,” “computer-storage media,” and “device-storagemedia” specifically exclude carrier waves, modulated data signals, andother such media, at least some of which are covered under the term“signal medium.”

“Component” refers to a device, physical entity, or logic havingboundaries defined by function or subroutine calls, branch points, APIs,or other technologies that provide for the partitioning ormodularization of particular processing or control functions. Componentsmay be combined via their interfaces with other components to carry outa machine process. A component may be a packaged functional hardwareunit designed for use with other components and a part of a program thatusually performs a particular function of related functions. Componentsmay constitute either software components (e.g., code embodied on amachine-readable medium) or hardware components. A “hardware component”is a tangible unit capable of performing certain operations and may beconfigured or arranged in a certain physical manner. In various exampleembodiments, one or more computer systems (e.g., a standalone computersystem, a client computer system, or a server computer system) or one ormore hardware components of a computer system (e.g., a processor or agroup of processors) may be configured by software (e.g., an applicationor application portion) as a hardware component that operates to performcertain operations as described herein. A hardware component may also beimplemented mechanically, electronically, or any suitable combinationthereof. For example, a hardware component may include dedicatedcircuitry or logic that is permanently configured to perform certainoperations. A hardware component may be a special-purpose processor,such as a field-programmable gate array (FPGA) or an applicationspecific integrated circuit (ASIC). A hardware component may alsoinclude programmable logic or circuitry that is temporarily configuredby software to perform certain operations. For example, a hardwarecomponent may include software executed by a general-purpose processoror other programmable processor. Once configured by such software,hardware components become specific machines (or specific components ofa machine) uniquely tailored to perform the configured functions and areno longer general-purpose processors. It will be appreciated that thedecision to implement a hardware component mechanically, in dedicatedand permanently configured circuitry, or in temporarily configuredcircuitry (e.g., configured by software), may be driven by cost and timeconsiderations. Accordingly, the phrase “hardware component” (or“hardware-implemented component”) should be understood to encompass atangible entity, be that an entity that is physically constructed,permanently configured (e.g., hardwired), or temporarily configured(e.g., programmed) to operate in a certain manner or to perform certainoperations described herein. Considering embodiments in which hardwarecomponents are temporarily configured (e.g., programmed), each of thehardware components need not be configured or instantiated at any oneinstance in time. For example, where a hardware component comprises ageneral-purpose processor configured by software to become aspecial-purpose processor, the general-purpose processor may beconfigured as respectively different special-purpose processors (e.g.,comprising different hardware components) at different times. Softwareaccordingly configures a particular processor or processors, forexample, to constitute a particular hardware component at one instanceof time and to constitute a different hardware component at a differentinstance of time. Hardware components can provide information to, andreceive information from, other hardware components. Accordingly, thedescribed hardware components may be regarded as being communicativelycoupled. Where multiple hardware components exist contemporaneously,communications may be achieved through signal transmission (e.g., overappropriate circuits and buses) between or among two or more of thehardware components. In embodiments in which multiple hardwarecomponents are configured or instantiated at different times,communications between such hardware components may be achieved, forexample, through the storage and retrieval of information in memorystructures to which the multiple hardware components have access. Forexample, one hardware component may perform an operation and store theoutput of that operation in a memory device to which it iscommunicatively coupled. A further hardware component may then, at alater time, access the memory device to retrieve and process the storedoutput. Hardware components may also initiate communications with inputor output devices, and can operate on a resource (e.g., a collection ofinformation). The various operations of example methods described hereinmay be performed, at least partially, by one or more processors that aretemporarily configured (e.g., by software) or permanently configured toperform the relevant operations. Whether temporarily or permanentlyconfigured, such processors may constitute processor-implementedcomponents that operate to perform one or more operations or functionsdescribed herein. As used herein, “processor-implemented component”refers to a hardware component implemented using one or more processors.Similarly, the methods described herein may be at least partiallyprocessor-implemented, with a particular processor or processors beingan example of hardware. For example, at least some of the operations ofa method may be performed by one or more processors 2102 orprocessor-implemented components. Moreover, the one or more processorsmay also operate to support performance of the relevant operations in a“cloud computing” environment or as a “software as a service” (SaaS).For example, at least some of the operations may be performed by a groupof computers (as examples of machines including processors), with theseoperations being accessible via a network (e.g., the Internet) and viaone or more appropriate interfaces (e.g., an API). The performance ofcertain of the operations may be distributed among the processors, notonly residing within a single machine, but deployed across a number ofmachines. In some example embodiments, the processors orprocessor-implemented components may be located in a single geographiclocation (e.g., within a home environment, an office environment, or aserver farm). In other example embodiments, the processors orprocessor-implemented components may be distributed across a number ofgeographic locations.

“Carrier Signal” refers to any intangible medium that is capable ofstoring, encoding, or carrying instructions for execution by themachine, and includes digital or analog communications signals or otherintangible media to facilitate communication of such instructions.Instructions may be transmitted or received over a network using atransmission medium via a network interface device.

“Computer-Readable Medium” refers to both machine-storage media andtransmission media. Thus, the terms include both storage devices/mediaand carrier waves/modulated data signals. The terms “machine-readablemedium,” “computer-readable medium” and “device-readable medium” meanthe same thing and may be used interchangeably in this disclosure. Insome embodiments, the computer-readable medium may be non-transitorycomputer-readable medium.

“Client Device” refers to any machine that interfaces to acommunications network to obtain resources from one or more serversystems or other client devices. A client device may be, but is notlimited to, a mobile phone, desktop computer, laptop, portable digitalassistants (PDAs), smartphones, tablets, ultrabooks, netbooks, laptops,multi-processor systems, microprocessor-based or programmable consumerelectronics, game consoles, set-top boxes, or any other communicationdevice that a user may use to access a network. In the subjectdisclosure, a client device is also referred to as an “electronicdevice.”

“Ephemeral Message” refers to a message that is accessible for atime-limited duration. An ephemeral message may be a text, an image, avideo and the like. The access time for the ephemeral message may be setby the message sender. Alternatively, the access time may be a defaultsetting or a setting specified by the recipient. Regardless of thesetting technique, the message is transitory.

What is claimed is:
 1. A method, comprising: accessing video datacomprising images of a presenter creating a tutorial, the tutorialdepicting the presenter applying a beauty product to a body part of thepresenter; processing the video data, using at least one processor, toidentify changes to the body part of the presenter from an applicationof the beauty product; receiving input from the presenter thatidentifies the beauty product applied to the body part; and associatingthe changes to the body part of the presenter with the beauty product.2. The method of claim 1 further comprising: storing, within a datastore, association data associating the beauty product with the changesto the body part.
 3. The method of claim 1, wherein the changes to thebody part of the presenter comprise a color change of the body part or achange of texture of the body part.
 4. The method of claim 1 furthercomprising: retrieving information regarding the beauty product; causingpresentation of information regarding the beauty product on a displaydevice; and storing, within a data store, association data associatingthe beauty product with the tutorial.
 5. The method of claim 4, whereinthe information regarding the beauty product comprises one or more of: aname of the beauty product, the body part associated with the beautyproduct, a manufacturer of the beauty product, a website associated withthe beauty product, an image of the beauty product, a product code ofthe beauty product, and an indication of ratings of the beauty product.6. The method of claim 1 wherein the video data is live video data. 7.The method of claim 1 further comprising: determining effects indicatingchanges to make to a second video data of a user to simulate applyingthe beauty product to the body part of the user, wherein the effects aredetermined based on the changes to the body part of the presenter; andstoring, within the data store, second association data associating theeffects with the beauty product and the tutorial.
 8. The method of claim7, wherein the changes to the body part of the presenter comprise one ormore of: a color, a texture, a transparency, a width, a length, a skintype for the body part, and a hair type of the body part.
 9. The methodof claim 1 wherein receiving input from the presenter that identifiesthe beauty product applied to the body part further comprises: recordingthe video data using a computing device; during the recording of thevideo data, receiving input from the presenter that identifies thebeauty product applied to the body part; and focusing a display, on thecomputing device, of the video data on images of the beauty product. 10.The method of claim 9, wherein the beauty product is a first beautyproduct and wherein the method further comprises: matching image data ofthe first beauty product in the images to a stored image data of asecond beauty product; and identifying the first beauty product as thesecond beauty product.
 11. The method of claim 1 further comprising:identifying a product code from an image of the beauty product; matchingthe product code to stored product codes to identify the beauty product;and retrieving information regarding the beauty product using theproduct code, wherein the product code is an identifier assigned to thebeauty product.
 12. The method of claim 11, wherein the informationregarding the beauty product comprises an identity of the beautyproduct, and wherein the method further comprises: causing presentationof a user interface item for the presenter to confirm or reject theidentity of the beauty product; and in response to receiving, via anapplication implemented by at least one processor, input from thepresenter that indicates rejection of the identity of the beautyproduct, recording new images of the presenter creating the tutorial,and identifying the beauty product based on the new images.
 13. Themethod of claim 1 wherein the input from the presenter is the presenterholding the beauty product within the video data.
 14. The method ofclaim 1, wherein the video data is live video data, and wherein themethod further comprising: in response to determining that the beautyproduct is within the live video data, causing an interrogation signalto be transmitted to the beauty product to solicit a radio frequencyidentification (RFID) identifying the beauty product, receiving radiosignals comprising the RFID from the beauty product, the RFID comprisinga product code, and identifying the beauty product using the productcode.
 15. The method of claim 1 further comprising: generating anidentifier for the beauty product, the identifier identifying the beautyproduct to an application recording the tutorial; causing presentationof information regarding the beauty product on a display device;receiving, input from the presenter, the input comprising custominformation regarding the beauty product; and storing a genericidentifier of the beauty product and the custom information based on thegenerated identifier.
 16. The method of claim 1, wherein the beautyproduct is associated with a step of the tutorial, wherein the tutorialcomprises a plurality of steps, and wherein the method furthercomprises: causing presentation of information regarding each step ofthe plurality of steps on a display device; and in response to receivingan indication of a selection of a step of the plurality of steps,associating the beauty product with the step.
 17. The method of claim 1further comprising: in response to a selection by the presenter,displaying a preview of a plurality of tutorials to the presenter,wherein each of the plurality of tutorials use the beauty product.
 18. Asystem comprising: at least one processor; and at least one memoryincluding instructions that, when executed by the at least oneprocessor, cause the at least one processor to perform operationscomprising: accessing video data comprising images of a presentercreating a tutorial, the tutorial depicting the presenter applying abeauty product to a body part of the presenter; processing the videodata, using at least one processor, to identify changes to the body partof the presenter from an application of the beauty product; receivinginput from the presenter that identifies the beauty product applied tothe body part; and associating the changes to the body part of thepresenter with the beauty product.
 19. The system of claim 18, whereinthe changes to the body part of the presenter comprise a color change ofthe body part or a change of texture of the body part.
 20. Anon-transitory computer-readable medium comprising instructions, whichwhen executed by a computing device comprising one or more processors,cause the one or more processors to: accessing video data comprisingimages of a presenter creating a tutorial, the tutorial depicting thepresenter applying a beauty product to a body part of the presenter;processing the video data, using at least one processor, to identifychanges to the body part of the presenter from an application of thebeauty product; receiving input from the presenter that identifies thebeauty product applied to the body part; and associating the changes tothe body part of the presenter with the beauty product.